Re: RFC: Kernel virtiofs driver

Hi Oleg,

Sorry for the long overdue reply, I missed this email. Yes, the three
diffs are D46295/D46296/D55047. I will also take a look, but can you
see if you can reproduce this with current HEAD? This looks like a bug
in the FUSE cached attributes locking that was overhauled this year.
so the logic for this may have been changed in the meantime.

Thanks,
Emil

On Tue, May 5, 2026 at 9:04 AM Oleg Sidorkin <osidorkin@gmail.com> wrote:
>
> пт, 13 февр. 2026 г. в 05:01, Emil Tsalapatis <emil@etsalapatis.com>:
> >
> > On Thu, Feb 12, 2026 at 9:01 AM Oleg Sidorkin <osidorkin@gmail.com> wrote:
> >>
> >> Hello.
> >>
> >> My favorite test to run in bhyve guest:
> >>
> >> root@:/usr/src # uname -a
> >> FreeBSD  16.0-CURRENT FreeBSD 16.0-CURRENT #1
> >> main-n283712-16c902f05853: Sat Feb  7 02:10:47 MSK 2026
> >> olsi@:/usr/obj/usr/src/amd64.amd64/sys/QUADKERNEL amd64
> >> root@:/usr/src # mount
> >> /dev/vtbd0p2 on / (ufs, local, soft-updates, journaled soft-updates)
> >> devfs on /dev (devfs)
> >> obj on /usr/obj (p9fs, local)
> >> 192.168.2.1:/vms/freebsd-current/usr/home on /usr/home (nfs)
> >> 192.168.2.1:/vms/freebsd-current/usr/src on /usr/src (nfs)
> >> 192.168.2.1:/vms/freebsd-current/usr/lib/debug on /usr/lib/debug (nfs)
> >> 192.168.2.1:/usr/ports/distfiles on /usr/ports/distfiles (nfs)
> >> root@:/usr/src # make -j4 buildworld buildkernel
> >>
> >> Panics guest system in a few minutes (everything is ok when /obj is
> >> mounted over nfs):
> >>
> >> db> bt
> >> Tracing pid 2248 tid 100165 td 0xfffff80100eb8780
> >> kdb_enter() at kdb_enter+0x33/frame 0xfffffe006833e560
> >> panic() at panic+0x43/frame 0xfffffe006833e5c0
> >> freevnode() at freevnode+0x2d5/frame 0xfffffe006833e620
> >> vput_final() at vput_final+0x96/frame 0xfffffe006833e670
> >> vfs_hash_insert() at vfs_hash_insert+0x226/frame 0xfffffe006833e6c0
> >> p9fs_vget_common() at p9fs_vget_common+0x39b/frame 0xfffffe006833e770
> >> p9fs_lookup() at p9fs_lookup+0x4ad/frame 0xfffffe006833e8c0
> >> VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0x57/frame 0xfffffe006833e8f0
> >> vfs_lookup() at vfs_lookup+0x5aa/frame 0xfffffe006833e980
> >> namei() at namei+0x35d/frame 0xfffffe006833e9e0
> >> kern_execve() at kern_execve+0x2d1/frame 0xfffffe006833ed80
> >> sys_execve() at sys_execve+0x54/frame 0xfffffe006833ee00
> >> amd64_syscall() at amd64_syscall+0x169/frame 0xfffffe006833ef30
> >> fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe006833ef30
> >> --- syscall (59, FreeBSD ELF64, execve), rip = 0x2cc73682f46a, rsp =
> >> 0x2cc731eb2028, rbp = 0x2cc731eb2170 ---
> >> db>
> >>
> >> I'll be really happy to test virtiofs and I hope it will do better.
> >>
> >
> > If you hit any bugs, let me know! I'd be happy to take a look. Please include some additional patches apart from the two diffs (https://github.com/etsal/freebsd-src/tree/virtiofs), that handle some issues virtiofs was triggering in the FUSE layer (D55047, D55046).
> >
> > Also, if you could try running with a debugging kernel I'd really appreciate it - it will help us better interpret any issues. For example, with a config like this:
> >
> > include GENERIC
> >
> > ident   DEBUG
> >
> > options         KDB                     # Enable kernel debugger support.
> > options         KDB_TRACE               # Print a stack trace for a panic.
> > options         DDB
> > options         DDB_NUMSYM
> > options         GDB
> >
> > options         INVARIANTS
> > options         INVARIANT_SUPPORT
> > options         DIAGNOSTIC
> >
> > # KTR
> > options         KTR
> > options         ALQ
> > options         KTR_ALQ
> >
> > # KASAN
> > options KASAN
> >
> >
> > Thanks,
> > Emil
> >
> >
> >
> >>
> >> Thanks
> >>
> >> вт, 10 февр. 2026 г. в 09:57, Mario Marietto <marietto2008@gmail.com>:
> >> >
> >> > Hello Emil,
> >> >
> >> > Inside a FreeBSD guest OS (15.0-RELEASE) I do :
> >> >
> >> > kldload virtio_p9fs
> >> >
> >> > kldload p9fs_load
> >> >
> >> > mount -t p9fs sharename /mnt/host
> >> >
> >> > This works for me,I can share files between FreeBSD 15.0 guest and FreeBSD 14.3 host os. So,what's missing in this case and which features you added ?
> >> >
> >> > Thanks.
> >> >
> >> >
> >> >
> >> > On Tue, Feb 10, 2026 at 4:05 AM Emil Tsalapatis <emil@etsalapatis.com> wrote:
> >> >>
> >> >> Hi everyone,
> >> >>
> >> >>     I recently finished the virtiofs driver and it is now ready for review. The device allows for sharing directories between a FreeBSD guest and a host.
> >> >>
> >> >> The driver really is two components:
> >> >>
> >> >> 1) The virtio device that sends FUSE tickets to and from the host: D46295
> >> >> 2) The file system that gets mounted in the guest: D46296.
> >> >>
> >> >> To test it you need a couple additional fixes/workarounds for FUSE-related issues. You can grab a working tree here or apply diffs D55047 and D55046. D55046 is a workaround, but still prevents an assertion failure related to FUSE caching until the underlying issue is properly fixed on HEAD.
> >> >>
> >> >> To use it, make sure you are creating virtiofs device on the host then from the FreeBSD guest run
> >> >>
> >> >> mount -t virtiofs <tag> <mountpoint>
> >> >>
> >> >> where <tag> is the name tag you gave to the virtiofs device in the host VMM.
> >> >>
> >> >> Reviews and testing welcome!
> >> >>
> >> >> Thanks,
> >> >> Emil
> >> >>
> >> >>
> >> >
> >> >
> >> > --
> >> > Mario.
> >>
> >>
> >>
> >> --
> >> Oleg Sidorkin
>
> Hi, I finally was able to set up an appropriate VM to test it.
> I applied 3 of 4 patches (all except
> https://reviews.freebsd.org/D55046 that seems to be already applied in
> a different way).
>
> It worked fine for simple file operations like creating and editing
> text files, but when I tried to use virtiofs mount as /usr/obj, it
> panicked almost instantly with the following backtrace
>
> #11 0xffffffff80bdc0a3 in panic (
>     fmt=0xffffffff81da22a0 <cnputs_mtx> "\017\344\"\201\377\377\377\377")
>     at /usr/src/sys/kern/kern_shutdown.c:887
> #12 0xffffffff82e3a7c9 in fuse_vnop_close (ap=<optimized out>)
>     at /usr/src/sys/fs/fuse/fuse_vnops.c:910
> #13 0xffffffff81211c22 in VOP_CLOSE_APV (vop=0xffffffff82e467a8 <fuse_vnops>,
>     a=a@entry=0xfffffe0068343ce8) at vnode_if.c:469
>         rc = <optimized out>
> #14 0xffffffff80b82411 in VOP_CLOSE (vp=<optimized out>, fflag=1, cred=0x12,
> #15 do_execve (td=0xfffff80100bc3000, args=0xfffffe0068343d98,
>     mac_p=<optimized out>, oldvmspace=<optimized out>)
>     at /usr/src/sys/kern/kern_exec.c:978
>
> (I can paste the entire stracktrace if needed).
>
> It looks like this assert:
> https://github.com/freebsd/freebsd-src/blob/dc00f118405e8638ceb13b288e14164a8a9ba669/sys/fs/fuse/fuse_vnops.c#L910
> added to recent -CURRENT, however I couldn't find where the lock
> should be taken.
>
>
> Thanks
>
> Oleg Sidorkin