nfsd server cache flooded, try to increase nfsrc_floodlevel

[Rick Macklem]

Harald Schmalzbauer wrote:
> Bezüglich Rick Macklem's Nachricht vom 08.08.2013 14:20 (localtime):
> > Lars Eggert wrote:
> >> Hi,
> >>
> >> every few days or so, my -STABLE NFS server (v3 and v4) gets
> >> wedged
> >> with a ton of messages about "nfsd server cache flooded, try to
> >> increase nfsrc_floodlevel" in the log, and nfsstat shows TCPPeak
> >> at
> >> 16385. It requires a reboot to unwedge, restarting the server does
> >> not help.
> >>
> >> The clients are (mostly) six -CURRENT nfsv4 boxes that netboot
> >> from
> >> the server and mount all drives from there.
> >>
Have you tried increasing vfs.nfsd.tcphighwater?
This needs to be increased to increase the flood level above 16384.

Garrett Wollman sets:
vfs.nfsd.tcphighwater=100000
vfs.nfsd.tcpcachetimeo=300

or something like that, if I recall correctly.

rick


> >> I googled around and saw that others have hit this issue, but I
> >> haven't seen any resolution posted. I guess I can increase
> >> NFSRVCACHE_FLOODLEVEL in the source, but I wonder if I wouldn't
> >> simply hit the increase value after a little while longer...
> >>
> >> Lars
> >>
> > You can either try this patch (which dynamically adjusts
> > nfsrc_floodlevel
> > along with handling a variety of overhead issues for the DRC under
> > heavy load):
> >    http://people.freebsd.org/~rmacklem/drc4.patch
> >
> > or just bump it up a bunch. The default value was safe for a server
> > with 256Mbytes
> > of ram and a default mbuf cluster limit. The only thing you might
> > have to do
> > along with bumping NFSRC_FLOODLEVEL up is increasing
> > kern.ipc.mbclusters.
> >
> > The variant of the above patch will make it into head someday, once
> > I merge
> > in changes from ivoras@'s similar patch and confer with him about
> > it.
> 
> Dear all,
> 
> regarding the conversation from last year - quoted above,
> I think I found the mentioned patch (it's variants) MFCd in r255532
> (from
> http://svnweb.freebsd.org/base?view=revision&revision=254337),
> so it's included in 9.3-RELEASE.
> 
> Unfortunately I'm still having the nfsrc_floodlevel problem with
> OpenOwner=16385, CacheSize=16385 (in nfsstat -e -s) in my production
> environment under 9.3-RELEASE-amd64.
> Extremely light load on the server (2 (FreeBSD8/9) clients), but the
> building client (nfsv4) locks up frequently. It mounts 'home' and
> 'ports/ports' via NFSv4 (this time, 'make index' in nfs-mounted
> /usr/ports killed the nfsv4server).
> 
> 
>     I found another interesting 3 years old patch/thread, which seems
> never beeing comitted:
> http://lists.freebsd.org/pipermail/freebsd-fs/2011-July/012016.html
> 
> I don't really understand all these details of nfs(v4), but I observe
> problems with regular usage, so I wanted to ask if there are new
> findings regarding the "nfsd server cache flooded, try to increase
> nfsrc_floodlevel" messages (while 'nfsrc_floodlevel' doesn't seem to
> be
> tunable in 9.3).
> To my understanding, it's a problem on the server side, right?
> 
> Is the fix from 3 years back still adequate (does apply with view
> offsets only to 9.3)?
> 
> I'm currently testing 9.3-RELEASE+noopen.patch, but it usually took
> two
> or three days until the client locked up (hadn't looked for the
> reason
> before the last issue, nfs(v4) was brand new reintroduced here)
> 
> Thanks,
> 
> -Harry
> 
>