Re: FreeBSD 13.2 NFS client mount hangs

On Sun, Oct 1, 2023 at 8:01 PM J David <j.david.lists@gmail.com> wrote:
>
> On Sat, Sep 30, 2023 at 6:06 PM Rick Macklem <rick.macklem@gmail.com> wrote:
> > --> I'd suggest you try and disable delegations.  I do not know how to
> > do this on
> >      the Linux server, but not running the nfscbd(8) daemon should stop them
> >      from being issued.
>
> The nfscbd daemon is not running on any of the clients.
>
> > If the Linux server still
> > issues delegations
>
> How would I determine that?
nfsstat -E -c
and then look at the number under "Delegs". It is a current count of
delegations, so if it remains 0 over time, no delegations are being issued.

>
> > Hmm, interesting. 10068 is NFS4ERR_RETRY_UNCACHED_REP.
> > I have never seen (and do not recall anyone else reporting) this error
> > return.
>
> It was days earlier than this issue and *might* be related to someone
> rebooting an NFS server the client is connected to at that time. Hard
> to tell for sure without timestamps.
Yes. This would occur after a NFS server reboot or some other occurrence
that causes the client to do a TCP reconnect.

I have attached a small patch which should make the NFS client handle
this error correctly.

rick

>
> > I'd suggest you first check network connetcivity. Both NFS client and NFS
> > server should be able to ping each other.
>
> The network connectivity is OK; network interface error counts are
> monitored with munin on all systems, and don't appear to be an issue.
>
> On that specific system's dedicated interface for NFS:
>
> Name    Mtu Network       Address              Ipkts Ierrs Idrop
> Opkts Oerrs  Coll
> net1   1500 <Link#2>      (redacted) 18754677570     0     0
> 18598354347     0     0
>
> Also, in both cases (which occurred with different client & server
> pairs), there were three different directories from the server mounted
> on the client.
>
> One was affected. The other two worked fine. Listing files and
> directories. Locking. Calculating hashes of large files.
> Creating/writing files. Etc.
>
> E.g. /a /b and /c were mounted from nfs.server and /a and /b were fine
> while /c was hung.  (And, as I mentioned before but seems worth
> repeating here for completeness, other FreeBSD clients had /c mounted
> with no problems while it was hung on this machine.)
>
>
> > If that is the case, then I'd suggest you capture packets. On the FreeBSD
> > end:
> > # tcpdump -s 0 -w out.pcap host <nfs-server-name>
> > Let this run for a while and then pull out.pcap into wireshark and see what
> > traffic is going between the NFS client and server.
> > (Unlike tcpdump, wireshark does know how to decode NFS properly.)
>
> If/when the issue happens again, I will attempt to do this and report back.
>
> Thanks!