zfs recv causes nfs server to throw NFSERR_IO i/o errors
Daniel Genis
daniel.genis at gmx.de
Fri Aug 5 13:22:42 UTC 2016
Hi everyone,
we've been tracing an issue where snapshot replication is causing
interruptions for the NFS serivce.
The problem is as follows:
Every time a zfs recv finishes, there is a chance for the NFS server to
return an NFSERR_IO for a GETATTR call. This shows up as input/output
errors on the nfs clients.
Here the tcpdump showing the NFS conversation:
https://nopaste.me/view/95d1a79d
NFS 202 V3 GETATTR Call (Reply In 6043), FH: 0x8c711a60
NFS 98 V3 GETATTR Reply (Call In 6042) Error: NFS3ERR_IO
NFS 222 V3 LOOKUP Call (Reply In 6046), DH: 0x6694634f/example.file.txt
NFS 102 V3 LOOKUP Reply (Call In 6045) Error: NFS3ERR_ACCES
We've been able to verify that there is a _direct_ correlation between
the zfs recv command and these NFS errors. For every input/output error
we can find a log entry of a replication just finishing (zfs recv exiting).
The receiving server is running 10.3-RELEASE
I've read about a VFS/ZFS deadlock issue which is to be included/fixed
in Freebsd 11.0-BETA4.
Could our issue be related?
Otherwise does anyone have any suggestions how to tackle this issue?
For the record, say we have two volumes:
tank/volumeA and tank/volumeB
If there is a zfs recv busy for tank/volumeA then tank/volumeB can get
these NFS "io" errors, it does not have to be the same volume.
Has anyone else seen/experience this as well?
Any insights are appreciated!
With kind regards,
Daniel
More information about the freebsd-stable
mailing list