From nobody Sun Jul 14 07:11:26 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WMGj66Rmmz5QRBs for ; Sun, 14 Jul 2024 07:11:38 +0000 (UTC) (envelope-from theraven@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R10" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4WMGj661CPz4JfL; Sun, 14 Jul 2024 07:11:38 +0000 (UTC) (envelope-from theraven@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1720941098; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=xlSy/e5G2WS+8SvLwDomElrU0HhtI3CNGoNY+JJRCcw=; b=KPjNfkgIA++pVf0nQ/DVmJkKziQJXVVChSO+w/ZTSfXXIvdMf/BZO2GxjhKSYvqVUL8d2w 8acwEPYjE9//H7ArPgpEGgGfyNqvmjkiTEg7S5AjBWLwl0//ffUkEvsD9dS1+RXaYNsd1J kGl0pTlu99MTrkpTl4hfxe8rL7LgTG3IsnU8O9mBoxU7JBU+Xfnc5wLILMUw9wVHBCWV81 905N4jXpGbRiV48runhaASKfDJkWAO4GrqJlKWzvPlpxXkL+stLQSoaHmRrEtk6kCrzbDv VIjipNJSuQi+gfCmZdTu4L1WaR6Ge4ncCJ82Wu3DZvQSvnDDJNteGD8bMnMNtA== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1720941098; a=rsa-sha256; cv=none; b=FUw55+3n17E3Q+cyiBbLKmeD6aLnUD9znqQwIJODBRkek4tHMsRsEp5tgVk3rkemwF9KLp IRCsTt6IvRBT5aIIvpvkRPNBj5HegV5gnPvHdViAb5QrsSpAXl8djCTGRa0IA4ZD/L9IQy uwfcCDzd6IH8Jb20Z6VZwWK5rHUZOpSUzeHGjHaRMJuqRvidZEsksHuHL9UYJdSjt2swkf tX8L8a9pqycUkW5z0iDEvhhHzPMjc41GeCk3DjBU5bBeddw1c1Vh6RQqBtXTxJhODwDhaK d09q57gVWEXvv+2p3bT7KassR1uzut5pfOMy6yIn79sTuuxOxSEw012XM7wnzg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1720941098; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=xlSy/e5G2WS+8SvLwDomElrU0HhtI3CNGoNY+JJRCcw=; b=HlJBJ/8YEFCvyGF4Pi4jcjXGl7bSHRwZ3S05tYa0Sz0QrIAdRqHpPXRZXBdVl5Bq/793Lw 4D63ZfEhnG7xnj3idkn16lkQbYwbACrg4yvecCNKleky4pqeUL6gCBh9fLLr3howe4lgn4 DJdmHpydxkRMG77ri1Tlpeyk823q3+ayWAlT2kr3OY59UF4Y28T8I/9sYJXscjJgjHohgO WKmTCtxo+n08mmsUl5ORdi0gfFOpydSaT6/7d75ZwYglAKlCkzLx4YzQn/32O1HYj2a3hW x0SMOShi9xicOMCEXh8ASY9h+5gmyd7TiiBvDEkuZSdkfb2YlJvPPI6fVoJ2qg== Received: from smtp.theravensnest.org (smtp.theravensnest.org [45.77.103.195]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: theraven) by smtp.freebsd.org (Postfix) with ESMTPSA id 4WMGj65QbrzQ1l; Sun, 14 Jul 2024 07:11:38 +0000 (UTC) (envelope-from theraven@FreeBSD.org) Received: from smtpclient.apple (host86-138-165-11.range86-138.btcentralplus.com [86.138.165.11]) by smtp.theravensnest.org (Postfix) with ESMTPSA id 4FCF6442B; Sun, 14 Jul 2024 08:11:37 +0100 (BST) From: David Chisnall Message-Id: Content-Type: multipart/alternative; boundary="Apple-Mail=_E777BA92-4526-42F6-81C1-95C53597DD06" List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.600.62\)) Subject: Re: Is anyone working on VirtFS (FUSE over VirtIO) Date: Sun, 14 Jul 2024 08:11:26 +0100 In-Reply-To: Cc: Alan Somers , FreeBSD Hackers To: Warner Losh References: X-Mailer: Apple Mail (2.3774.600.62) --Apple-Mail=_E777BA92-4526-42F6-81C1-95C53597DD06 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Wow, that looks incredibly useful. Not needing bhyve / qemu (nested, if = your main development is a VM) to test virtio drivers would be a huge = productivity win. =20 David > On 13 Jul 2024, at 23:06, Warner Losh wrote: >=20 > Hey David, >=20 > You might want to check out https://reviews.freebsd.org/D45370 which = has the testing framework as well as hints at other work that's been = done for virtiofs by Emil Tsalapatis. It looks quite interesting. = Anything he's done that's at odds with what I've said just shows where = my analysis was flawed :) This looks quite promising, but I've not had = the time to look at it in detail yet. >=20 > Warner >=20 > On Sat, Jul 13, 2024 at 2:44=E2=80=AFAM David Chisnall = > wrote: >> On 31 Dec 2023, at 16:19, Warner Losh > wrote: >>>=20 >>> Yea. The FUSE protocol is going to be the challenge here. For this = to be useful, the VirtioFS support on the FreeBSD needs to be 100% in = the kernel, since you can't have userland in the loop. This isn't so = terrible, though, since our VFS interface provides a natural breaking = point for converting the requests into FUSE requests. The trouble, I = fear, is a mismatch between FreeBSD's VFS abstraction layer and Linux's = will cause issues (many years ago, the weakness of FreeBSD VFS caused = problems for a company doing caching, though things have no doubt = improved from those days). Second, there's a KVM tie-in for the direct = mapped pages between the VM and the hypervisor. I'm not sure how that = works on the client (FreeBSD) side (though the description also says = it's mapped via a PCI bar, so maybe the VM OS doesn't care). >>=20 >> =46rom what I can tell from a little bit of looking at the code, our = FUSE implementation has a fairly cleanly abstracted layer (in = fuse_ipc.c) for handling the message queue. For VirtioFS, it would = 'just' be necessary to factor out the bits here that do uio into = something that talked to a VirtIO ring. I don=E2=80=99t know what the = VFS limitations are, but since the protocol for VirtioFS is the kernel = <-> userspace protocol for FUSE, it seems that any functionality that = works with FUSE filesystems in userspace would work with VirtioFS = filesystems. >>=20 >> The shared buffer cache bits are nice, but are optional, so could be = done in a later version once the basic functionality worked. =20 >>=20 >> David >>=20 --Apple-Mail=_E777BA92-4526-42F6-81C1-95C53597DD06 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8 Wow, that = looks incredibly useful.  Not needing bhyve / qemu (nested, if your = main development is a VM) to test virtio drivers would be a huge = productivity win.  

David

On 13 Jul 2024, at 23:06, Warner Losh = <imp@bsdimp.com> wrote:

Hey = David,

You might want to check out  https://reviews.freebsd.org/D4= 5370 which has the testing framework as well as hints at other work = that's been done for virtiofs by Emil Tsalapatis. It looks = quite interesting. Anything he's done that's at odds with what I've said = just shows where my analysis was flawed :) This looks quite promising, = but I've not had the time to look at it in detail = yet.

Warner

On Sat, Jul = 13, 2024 at 2:44=E2=80=AFAM David Chisnall <theraven@freebsd.org> = wrote:
On 31 Dec 2023, at 16:19, Warner = Losh <imp@bsdimp.com> wrote:

Yea. The FUSE protocol is going to be the challenge = here. For this to be useful, the VirtioFS support on the = FreeBSD  needs to be 100% in the kernel, since you can't have = userland in the loop. This isn't so terrible, though, since our VFS = interface provides a natural breaking point for converting the requests = into FUSE requests. The trouble, I fear, is a mismatch between FreeBSD's = VFS abstraction layer and Linux's will cause issues (many years ago, the = weakness of FreeBSD VFS caused problems for a company doing caching, = though things have no doubt improved from those days). Second, there's a = KVM tie-in for the direct mapped pages between the VM and the = hypervisor. I'm not sure how that works on the client (FreeBSD) side = (though the description also says it's mapped via a PCI bar, so maybe = the VM OS doesn't care).

=46rom = what I can tell from a little bit of looking at the code, our FUSE = implementation has a fairly cleanly abstracted layer (in fuse_ipc.c) for = handling the message queue.  For VirtioFS, it would 'just' be = necessary to factor out the bits here that do uio into something that = talked to a VirtIO ring.  I don=E2=80=99t know what the VFS = limitations are, but since the protocol for VirtioFS is the kernel = <-> userspace protocol for FUSE, it seems that any functionality = that works with FUSE filesystems in userspace would work with VirtioFS = filesystems.

The shared buffer cache bits are = nice, but are optional, so could be done in a later version once the = basic functionality worked. =  

David


= --Apple-Mail=_E777BA92-4526-42F6-81C1-95C53597DD06--