Re: FreeBSD 15.0 nfsd makes all write accesses EPERM

On Wed, Jan 7, 2026 at 5:22 AM Rick Macklem <rick.macklem@gmail.com> wrote:
>
> On Wed, Jan 7, 2026 at 2:46 AM Aurélien Couderc
> <aurelien.couderc2002@gmail.com> wrote:
> >
> > On Tue, Jan 6, 2026 at 11:40 PM Xin LI <delphij@gmail.com> wrote:
> > >
> > >
> > >
> > > On Tue, Jan 6, 2026 at 12:37 PM Aurélien Couderc <aurelien.couderc2002@gmail.com> wrote:
> > >>
> > >> On Tue, Jan 6, 2026 at 9:07 PM Aurélien Couderc
> > >> <aurelien.couderc2002@gmail.com> wrote:
> > >> >
> > >> > Our CI uses this script to quickly setup a FreeBSD NFSv4.1 server for
> > >> > testing. The script worked with FreeBSD 14.2, but with FreeBSD 15.0
> > >> > all files are suddenly read-only (write access gets EPERM):
> > >> >
> > >> > sysrc rpcbind_enable="YES"
> > >> > sysrc mountd_enable="YES"
> > >> > sysrc nfs_server_enable=YES
> > >> > sysrc nfsv4_server_enable=YES
> > >> > sysrc nfsuserd_flags="-domain GLOBAL.LOC"
> > >> > sysrc nfsuserd_enable=YES
> > >> > sysrc nfscbd_enable=YES
> > >> > printf 'vfs.nfs.enable_uidtostring=0\n' >>'/etc/sysctl.conf'
> > >> > printf 'vfs.nfsd.enable_stringtouid=0\n' >>'/etc/sysctl.conf'
> > >> > printf 'vfs.nfsd.issue_delegations=1\n' >>'/etc/sysctl.conf'
> > >> > mkdir /nfsdata
> > >> > chmod a+rwxt /nfsdata
> > >> > printf 'V4: /\n' >'/etc/exports'
> > >> > printf '/nfsdata -network=80.247.17.0 -mask=255.255.255.0  -sec=sys\n'
> > >> > >>'/etc/exports'
> > >> > service nfsd start
> > >> >
> > >> >
> > >> > The failed NFS4.2 op looks like this in wireshark:
> > >> >
> > >> > Network File System, Ops(8): SEQUENCE, PUTFH, SAVEFH, OPEN, GETFH,
> > >> > GETATTR, RESTOREFH, GETATTR
> > >> >     [Program Version: 4]
> > >> >     [V4 Procedure: COMPOUND (1)]
> > >> >     Tag: open
> > >> >     minorversion: 2
> > >> >     Operations (count: 8): SEQUENCE, PUTFH, SAVEFH, OPEN, GETFH,
> > >> > GETATTR, RESTOREFH, GETATTR
> > >> >         Opcode: SEQUENCE (53)
> > >> >         Opcode: PUTFH (22)
> > >> >         Opcode: SAVEFH (32)
> > >> >         Opcode: OPEN (18)
> > >> >             seqid: 0x00000000
> > >> >             share_access: OPEN4_SHARE_ACCESS_WRITE (2)
> > >> >             share_deny: OPEN4_SHARE_DENY_NONE (0)
> > >> >             clientid: 0x0000000000000000
> > >> >             owner: <DATA>
> > >> >             Open Type: OPEN4_CREATE (1)
> > >> >                 Create Mode: UNCHECKED4 (0)
> > >> >                 Attr mask[0]: 0x02000010 (Size, Hidden)
> > >> >                     reqd_attr: Size (4)
> > >> >                         size: 0
> > >> >                     reco_attr: Hidden (25)
> > >> >                         fattr4_hidden: No
> > >> >                 Attr mask[1]: 0x00004002 (Mode, System)
> > >> >                     reco_attr: Mode (33)
> > >> >                         mode: 0644, Name: Unknown, Read permission for
> > >> > owner, Write permission for owner, Read permission for group, Read
> > >> > permission for others
> > >> >                     reco_attr: System (46)
> > >> >                         fattr4_system: No
> > >> >             Claim Type: CLAIM_NULL (0)
> > >>
> > >> I think I found the problem: If our JAVA client does not set the
> > >> FATTR_HIDDEN and FATTR_SYSTEM attributes, then write access works.
> > >>
> > >> Why did this change in FreeBSD?
> > >
> > >
> > > I think support for DOS "hidden" and "system" attributes was implemented in c5d72d29fe0e1a6ac1aaf622d8c4a6ca9422517a (this is in the NFSv4.1 specification) and ZFS supports both attributes via named attributes and therefore reports _PC_HAS_HIDDENSYSTEM and NFS server uses that to decide whether to report this feature.
> >
> > The problem with FATTR4_HIDDEN and FATTR4_SYSTEM is unrelated to NFS
> > named attributes, it just happened that I tested our JAVA NFSV4.2
> > client with both.
> >
> > > I'm not sure why this translates to EPERM for writes, however.
> As noted above, it appears that ZFS returns EPERM when non-root
> tries to set an va_flags (system and hidden are va_flags) when done
> by non-root. Why the implementer chose EPERM, I do not know.
> --> Some va_flags can only be set by root and the bug is that the
>      current code assumes all va_flags can only be set by root.
> (At least that is what I think is the case from looking at the code.)
>
> As I said, a patch can easily confirm if this is the bug, which
> was why I asked if you can build a patched kernel to test this?
> (I can construct a test case, but that will take a while.)
When I "*fixed* ZFS to not return EPERM when the hidden,
system flags were being set (to false in this case, actually),
it panic'd.

So, instead, I came up with a patch that avoids setting
hidden, system in VOP_CREATE() and, instead, does them
via VOP_SETATTR() later in the processing of the NFSv4 OPEN.
This patch seems to work for my test case.

The patch is attached.  If you can test it, that would be
appreciated. Unless I hear it doesn't fix the problem,
this patch should be in 15.1 when it is released.

rick
ps: If you set up a problem report in bugs.freebsd.org,
      I will put the patch there as an attachment as well.

>
> Also, please create a problem report at bugs.freebsd.org
> for this, to keep track of it.
>
> rick
>
> >
> > Rick?
> >
> > Aurélien
> > --
> > Aurélien Couderc <aurelien.couderc2002@gmail.com>
> > Big Data/Data mining expert, chess enthusiast
> >