NFS+ZFS+nullfs on Server random Permission Denied errors on client (nfsv4)

[Bryan Drewery]

Server:
  9.1-RELEASE
  mount:
> tank/distfiles/freebsd on /tank/distfiles/freebsd (zfs, NFS exported, local, noatime, noexec, nfsv4acls)
/etc/zfs/exports:
> /tank/distfiles/freebsd -maproot=root -network 10.10.0.0/16

Clients:

8.3-RELEASE
fstab:
> tank:/tank/distfiles/freebsd                   /mnt/distfiles                  nfs             rw,bg,noatime,intr,rsize=65536,wsize=65536,readahead=8,nfsv4    0       0

HEAD r250991:
> tank:/tank/distfiles/freebsd              /mnt/distfiles    nfs             rw,bg,noatime,intr,rsize=65536,wsize=65536,readahead=8,nfsv4    0       0

9.1-STABLE r247421:
> tank:/tank/distfiles/freebsd              /mnt/distfiles    nfs             rw,bg,noatime,intr,rsize=65536,wsize=65536,readahead=8,nfsv4    0       0

Problem:

For months I constantly get random Permission Denied errors on the
client side. Just trying the read again can fix the problem. The client
will usually show this in dmesg as the same time:
> nfsv4 client/server protocol prob err=10020

Example errors:
> root at c1100_1:/usr/local/poudriere/data/logs/bulk/91amd64-default/2013-05-29_19:17:45/logs/errors # grep "Permission denied" *
> dconf-0.12.1_1.log:cannot open gnome3/dconf-0.12.1.tar.xz: Permission denied
> p5-Pod-Tests-1.19_1.log:pax: Unable to open /mnt/distfiles//Pod-Tests-1.19.tar.gz to read <Permission denied>
> sofia-sip-1.12.11.log:pax: Unable to open /mnt/distfiles//sofia-sip-1.12.11.tar.gz to read <Permission denied>
> tex-ptex-3.3_1.log:fetch: texlive-20120701-texmf.tar.xz: Permission denied
> tkman-2.2_2.log:pax: Unable to open /mnt/distfiles//tkman-2.2.tar.gz to read <Permission denied>


I have realized that this problem manifests when I nullfs mount the
/tank/distfiles/freebsd directory on the server and start doing reads
from it, while also reading from the clients.

It became so bad that I had to move my mail off of NFS and back onto the
mail server itself since I had questions of where mail had disappeared to!

Has anyone else experienced this? Is is fixed in STABLE already?


Bryan Drewery

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 899 bytes
Desc: OpenPGP digital signature
URL: <http://lists.freebsd.org/pipermail/freebsd-fs/attachments/20130529/4df4c74d/attachment.sig>