Slow disk write speeds over network

[Sean Chittenden]

> > net.inet.tcp.sendspace=65536
> 
> Double the default.  Might not be a good idea, unless you have a ton
> of memory.  You will potentially use 64K send + 64K receive times
> number of sockets.  Assuming 4G and near-perfect tuning, you will be
> limited to 16384 simultaneous connections fully packed before memory
> pressure causes your machine to crash.  I tend to like smaller
> buffers and more connections.  If you only have 512M, drop this
> number to 2048 simultaneous connections if all buffers are full.

4GB of memory in this box and it's not directly available to the
Internet: a very important point to take into note when tuning.

> > kern.maxfiles=65536
> 
> Seems kind of overkill for the number of connections you can support
> without overcommit, and the number of client machines you say you
> have.

This machine has a busy database on it too.  :)

> > net.inet.udp.recvspace=65536
> > net.inet.udp.maxdgram=57344
> 
> These are important for UDP NFS.  I do not reccomend it.

Other than I do use UDP NFS, so I do recommend these values.  :) The
relevance of the UDP vs TCP NFS should be kept to the archives, we
don't need to re-open that bikeshed.  :)

> > vfs.nfs.async=1
> 
> This is very dangerous, if you care about your data.  It permits
> NFS to ACK writes before they have been committed to stable
> storage.  With a large enough window size, this should not be
> necessary.

It's just as dangerous as write caching, same argument for/against
write caching, this is no worse.

> > net.inet.udp.log_in_vain=1
> 
> This is just overhead; I reccomend turning it off.

Ehh....  there's something floating around someplace where I've been
getting errant NFS UDP packets and I haven't figured out what's going
on... this is useful for debugging in the absence of a firewall.

> > net.inet.icmp.icmplim=20000
> 
> This is only useful for TCP; but it can be very useful.  Basically,
> this is "connection rate limiting".  If you have a ton of clients,
> or trying to "netbench" the system, then set this number up.  For
> 100 NFS clients, it likely does not matter.

I beg to differ... any ideas on how you throttle UDP connections
without ICMP?  A busy UDP NFS server will require this to be set
higher.

-sc

-- 
Sean Chittenden