bad NFS/UDP performance

[Jeremy Chadwick]

On Fri, Sep 26, 2008 at 04:35:17PM +0300, Danny Braniss wrote:
> > On Fri, Sep 26, 2008 at 12:27:08PM +0300, Danny Braniss wrote:
> > > > On Fri, Sep 26, 2008 at 10:04:16AM +0300, Danny Braniss wrote:
> > > > > Hi,
> > > > > 	There seems to be some serious degradation in performance.
> > > > > Under 7.0 I get about 90 MB/s (on write), while, on the same machine
> > > > > under 7.1 it drops to 20!
> > > > > Any ideas?
> > > > 
> > > > 1) Network card driver changes,
> > > could be, but at least iperf/tcp is ok - can't get udp numbers, do you
> > > know of any tool to measure udp performance?
> > > BTW, I also checked on different hardware, and the badness is there.
> > 
> > According to INDEX, benchmarks/iperf does UDP bandwidth testing.
> 
> I know, but I get about 1mgb, which seems somewhat low :-(
> 
> > 
> > benchmarks/nttcp should as well.
> > 
> > What network card is in use?  If Intel, what driver version (should be
> > in dmesg).
> 
> bge: <Broadcom NetXtreme Gigabit Ethernet Controller, ASIC rev. 0x9003> 
> and
> bce: <Broadcom NetXtreme II BCM5708 1000Base-T (B2)>
> and intels, but haven't tested there yet.

Both bge(4) and bce(4) claim to support checksum offloading.  You might
try disabling it (ifconfig ... -txcsum -rxcsum) to see if things
improve.  If not, more troubleshooting is needed.  You might also try
turning off TSO if it's supported (check your ifconfig output for TSO in
the options=<> section.  Then use ifconfig ... -tso)

> > Do you have any NFS-related tunings in /etc/rc.conf or /etc/sysctl.conf?
> > 
> no, but diffing the sysctl show:
> 
> 	-vfs.nfs.realign_test: 22141777
> 	+vfs.nfs.realign_test: 498351
> 
> 	-vfs.nfsrv.realign_test: 5005908
> 	+vfs.nfsrv.realign_test: 0
> 
> 	+vfs.nfsrv.commit_miss: 0
> 	+vfs.nfsrv.commit_blks: 0
> 
> changing them did nothing - or at least with respect to nfs throughput :-)

I'm not sure what any of these do, as NFS is a bit out of my league.
:-)  I'll be following this thread though!

-- 
| Jeremy Chadwick                                jdc at parodius.com |
| Parodius Networking                       http://www.parodius.com/ |
| UNIX Systems Administrator                  Mountain View, CA, USA |
| Making life hard for others since 1977.              PGP: 4BD6C0CB |