Initial NFS Test: Linux vs FreeBSD (769% slower)

[Marc G. Fournier]

Okay, I just bit the bullet and installed a Linux/Centos 6.4 system for testing purposes.  

All hardware is *identical*, except that the FreeBSD server has the nicer Intel card in it … the Linux is just using the onboard Broadcom.   Everything else is the same … same CPUs, same amount of memory, same switch, same netapp … the netapp only has one filer head, so its the same head.  I created two separate volumes, but both are on the same aggregate and the same size.

OSs:

FreeBSD 9.1-STABLE #2: Thu Apr 25 18:18:18 UTC 2013
	- custom kernel, I try to compile everything I need in, what I missed:

vs

Linux server03.openstudent.ca 2.6.32-358.6.1.el6.x86_64 #1 SMP Tue Apr 23 19:29:00 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux
	- CentOS 6.4 + 'yum update'


Everything is on the local drive *except* that I put /usr/local/jboss on the NFS drive, and there is no virtualization happening anywhere … everything is on the base server OS, and nothing else is running on either of the machines.


FreeBSD start time for jBoss .. 270731ms (768% *slower*)
Linux start time for jBoss ….…. 35218 ms 

Even hitting ctl-c after the test to stop the process is 2x faster:

FreeBSD: 4207ms (169% slower)
Linux: 2486ms

Note that both systems have an equal lack of tuning … I wouldn't even know *where* to start tuning Linux …

Here is nfsstat -m for the two systems … both using nfsv3, same rsize/wsize values, all default out of the box ...

FreeBSD:
192.168.1.1:/vol/freebsd_jboss on /usr/local/jboss-as-7.1.1.Final
nfsv3,tcp,resvport,hard,cto,lockd,sec=sys,acdirmin=3,acdirmax=60,acregmin=5,acregmax=60,nametimeo=60,negnametimeo=60,rsize=65536,wsize=65536,readdirsize=32768,readahead=1,wcommitsize=5175966,timeout=120,retrans=2

Linux:
/usr/local/jboss-as-7.1.1.Final from 192.168.1.1:/vol/linux_jboss
 Flags:	rw,relatime,vers=3,rsize=65536,wsize=65536,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=192.168.1.1,mountvers=3,mountport=4046,mountproto=udp,local_lock=none,addr=192.168.1.1


Note, I have time on my side here … we are doing a limit pilot over the next month, but it isn't until Sep/Oct that this project goes live … what can we do to narrow down that gap? 

Thx