serious networking (em) performance (ggate and NFS) problem

[Robert Watson]

On Thu, 18 Nov 2004, Wilko Bulte wrote:

> On Thu, Nov 18, 2004 at 12:27:44PM +0000, Robert Watson wrote..
> > 
> > On Wed, 17 Nov 2004, Emanuel Strobl wrote:
> > 
> > > I really love 5.3 in many ways but here're some unbelievable transfer
> > > rates, after I went out and bought a pair of Intel GigaBit Ethernet
> > > Cards to solve my performance problem (*laugh*): 
> > 
> > I think the first thing you want to do is to try and determine whether the
> > problem is a link layer problem, network layer problem, or application
> > (file sharing) layer problem.  Here's where I'd start looking:
> 
> And you definitely want to look at polling(4) 

He did, but he set the HZ at 256, which is sufficiently low as to
guarantee a substantial increase in latency, and likely guarantee
interface and socket queue overruns (although I haven't done the math to
verify that is the case).  Between the very finite sizes of ifnet send
queues, socket buffers, and if_em descriptors, and on-board buffers on the
card, high latency polling can result in lots of packet loss and delay
under load.  Hence the recommendation of a relatively high value of HZ so
that the queues in the driver are drained regularly, and sends
acknowledged so that the sent mbufs can be reclaimed and reused.

Robert N M Watson             FreeBSD Core Team, TrustedBSD Projects
robert at fledge.watson.org      Principal Research Scientist, McAfee Research