NFS client slow on amd64 6.2-PRERELEASE #2

[Heinrich Rebehn]

Jeremy Chadwick wrote:
> On Thu, Oct 05, 2006 at 05:28:03PM +0200, Heinrich Rebehn wrote:
>> root at antsrv1 [/tmp] # time dd if=/dev/zero of=/mnt/x/10MB.dat bs=1M count=10
>> 10485760 bytes transferred in 4.967248 secs (2110980 bytes/sec)
>>
>> root at antsrv1 [/tmp] # time dd if=/dev/zero of=/mnt/x/100MB.dat bs=1M count=100
>> 104857600 bytes transferred in 69.020366 secs (1519227 bytes/sec)
>>
>> root at antsrv1 [/tmp] # time dd if=/dev/zero of=/mnt/x/10MB.dat bs=1M count=10
>> 10485760 bytes transferred in 5.289492 secs (1982376 bytes/sec)
>>
>> root at antsrv1 [/tmp] # time dd if=/dev/zero of=/mnt/x/100MB.dat bs=1M count=100
>> 104857600 bytes transferred in 58.715595 secs (1785856 bytes/sec)
> 
> This isn't a valid test of performance, in my opinion.  bs=1M is a
> bad idea.  You shouldn't be using dd for this kind-of test at all.
> I should also note that bs=1M is not the same thing as obs=1M ibs=1M.
> dd's weird like that (someone can explain it, I'm sure -- I've seen
> it discussed in the pasT).

Ok, let's try cp:

root at antsrv1 [~] # mount_nfs -U -L antsrv2:/export/disk2 /mnt
root at antsrv1 [/tmp] # time cp 10MB.dat /mnt/x

real    0m6.884s
user    0m0.000s
sys     0m0.027s
root at antsrv1 [/tmp] # time cp 100MB.dat /mnt/x

real    1m8.744s
user    0m0.000s
sys     0m0.251s

Again, this yields less than 2MB/sec :-(

> 
> scp is an ""acceptable"" real-life test (not the best, but it's
> legitimate), and you only did it from antsrv1 --> antsrv2, not the
> other direction.  It's really too bad the OpenBSD guys refuse to
> incorporate the HP (high-performance) patches into OpenSSH, and
> being able to say "-c none" would *really* help when it comes to
> benchmarking network I/O via scp (in our case, we do dump over ssh
> across a segregated private LAN -- the encryption overhead slows
> our backups down to a crawl, and is worthless in our environment
> since as I said, segregated private LAN...)

So "-c none" would then have given even better performance compared to nfs.
> 
> That said:
> 
> I have seen cases where network peformance on BSD works fantastic
> uni-directionally -- for example, using FTP to "get" a file from
> a FreeBSD box on a 100mbit LAN results in a speed of about
> 300-400kbit/sec, while doing a "put" to the same box results in
> 90mbit/sec.
> 
> The problem in that case turned out to be duplex-related.  Both
> boxes were auto-negotiating with the Cisco switch correctly, and
> indeed the Cisco labelled them as auto-100/full, but as anyone who
> is familiar with Ciscos knows, auto-negotiation on Catalysts is
> far from reliable.  Both boxes reported auto-neg and being at 100/full
> as well.  I ended up hard-setting the boxes to use 100/full, and
> set the switch ports to 100/full, then rebooted both boxes (yes,
> this is sometimes required, as driver auto-neg code is a bit tweaky);
> voila, problem fixed.
> 
> If you can connect these two boxes directly via a crossover cable,
> and you still see the problems, then yes, there's something definitely
> amiss that needs investigating.  :-)
> 

I hardcoded both machines to 1000baseTX full duplex and rebooted both:
root at antsrv1 [/tmp] # ifconfig em0
em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
         options=b<RXCSUM,TXCSUM,VLAN_MTU>
         inet 134.102.176.16 netmask 0xffffff00 broadcast 134.102.176.255
         ether 00:04:23:d4:94:89
         media: Ethernet 1000baseTX <full-duplex>
         status: active
root at antsrv2 [/tmp] # ifconfig em0
em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
         options=b<RXCSUM,TXCSUM,VLAN_MTU>
         inet 134.102.176.17 netmask 0xffffff00 broadcast 134.102.176.255
         ether 00:0e:0c:ba:79:a8
         media: Ethernet 1000baseTX <full-duplex>
         status: active

The switch is a "dumb" LinkPro GB switch with nothing to configure.
How can i connect both boxes via a crossover cable and at the same time 
connect them to our lan? ANTSRV1 is our production server that i cannot 
take off line!
-- 

Heinrich Rebehn

University of Bremen
Physics / Electrical and Electronics Engineering
- Department of Telecommunications -

Phone : +49/421/218-4664
Fax   :            -3341