bad NFS/UDP performance

[Danny Braniss]

> On Fri, Sep 26, 2008 at 12:27:08PM +0300, Danny Braniss wrote:
> > > On Fri, Sep 26, 2008 at 10:04:16AM +0300, Danny Braniss wrote:
> > > > Hi,
> > > > 	There seems to be some serious degradation in performance.
> > > > Under 7.0 I get about 90 MB/s (on write), while, on the same machine
> > > > under 7.1 it drops to 20!
> > > > Any ideas?
> > > 
> > > 1) Network card driver changes,
> > could be, but at least iperf/tcp is ok - can't get udp numbers, do you
> > know of any tool to measure udp performance?
> > BTW, I also checked on different hardware, and the badness is there.
> 
> According to INDEX, benchmarks/iperf does UDP bandwidth testing.

I know, but I get about 1mgb, which seems somewhat low :-(

> 
> benchmarks/nttcp should as well.
> 
> What network card is in use?  If Intel, what driver version (should be
> in dmesg).

bge: <Broadcom NetXtreme Gigabit Ethernet Controller, ASIC rev. 0x9003> 
and
bce: <Broadcom NetXtreme II BCM5708 1000Base-T (B2)>
and intels, but haven't tested there yet.

> 
> > > 2) This could be relevant, but rwatson@ will need to help determine
> > >    that.
> > >    http://lists.freebsd.org/pipermail/freebsd-stable/2008-September/045109.html
> > 
> > gut feeling is that it's somewhere else:
> > 
> > Writing 16 MB file
> > 	BS    Count /---- 7.0 ------/ /---- 7.1 -----/
> >        1*512  32768 0.16s  98.11MB/s  0.43s 37.18MB/s
> >        2*512  16384 0.17s  92.04MB/s  0.46s 34.79MB/s
> >        4*512   8192 0.16s 101.88MB/s  0.43s 37.26MB/s
> >        8*512   4096 0.16s  99.86MB/s  0.44s 36.41MB/s
> >       16*512   2048 0.16s 100.11MB/s  0.50s 32.03MB/s
> >       32*512   1024 0.26s  61.71MB/s  0.46s 34.79MB/s
> >       64*512    512 0.22s  71.45MB/s  0.45s 35.41MB/s
> >      128*512    256 0.21s  77.84MB/s  0.51s 31.34MB/s
> >      256*512    128 0.19s  82.47MB/s  0.43s 37.22MB/s
> >      512*512     64 0.18s  87.77MB/s  0.49s 32.69MB/s
> >     1024*512     32 0.18s  89.24MB/s  0.47s 34.02MB/s
> >     2048*512     16 0.17s  91.81MB/s  0.30s 53.41MB/s
> >     4096*512      8 0.16s 100.56MB/s  0.42s 38.07MB/s
> >     8192*512      4 0.82s  19.56MB/s  0.80s 19.95MB/s
> >    16384*512      2 0.82s  19.63MB/s  0.95s 16.80MB/s
> >    32768*512      1 0.81s  19.69MB/s  0.96s 16.64MB/s
> > 
> > Average:                   75.86            33.00
> > 
> > the nfs filer is a NetWork Appliance, and is in use, so i get fluctuations in 
> > the
> > measurements, but the relation are similar, good on 7.0, bad on 7.1
> 
> Do you have any NFS-related tunings in /etc/rc.conf or /etc/sysctl.conf?
> 
no, but diffing the sysctl show:

	-vfs.nfs.realign_test: 22141777
	+vfs.nfs.realign_test: 498351

	-vfs.nfsrv.realign_test: 5005908
	+vfs.nfsrv.realign_test: 0

	+vfs.nfsrv.commit_miss: 0
	+vfs.nfsrv.commit_blks: 0

changing them did nothing - or at least with respect to nfs throughput :-)

danny