NFS DRC size

[Garrett Wollman]

<<On Sat, 9 Mar 2013 11:27:32 -0500 (EST), Rick Macklem <rmacklem at uoguelph.ca> said:

> around the highwater mark basically indicates this is working. If it wasn't
> throwing away replies where the receipt has been ack'd at the TCP
> level, the cache would grow very large, since they would only be
> discarded after a loonnngg timeout (12hours unless you've changes
> NFSRVCACHE_TCPTIMEOUT in sys/fs/nfs/nfs.h).

That seems unreasonably large.

> Well, the DRC will try to cache replies until the client's TCP layer
> acknowledges receipt of the reply. It is hard to say how many replies
> that is for a given TCP connection, since it is a function of the level
> of concurrently (# of nfsiod threads in the FreeBSD client)
> in the client. I'd guess it's somewhere between 1<->20?

Nearly all our clients are Linux, so it's likely to be whatever Debian
does by default.

> Multiply that by the number of TCP connections from all clients and
> you have about how big the server's DRC will be. (Some clients use
> a single TCP connection for the client whereas others use a separate
> TCP connection for each mount point.)

The Debian client appears to use a single TCP connection for
everything.

So if I want to support 2,000 clients each with 20 requests in flight,
that would suggest that I need a DRC size of 40,000, which my
experience shows is not sufficient with even a much smaller number of
clients.

-GAWollman