Optimising NFS for system files

Bernard Dugas bernard at dugas-family.org
Wed Dec 31 09:29:55 UTC 2008


I am trying a memory disk on server to see the effect of hard drive 
performances, and also discovering the function :-)

The conclusion is that memory disk is faster that this drive ;-)
  45MB/s vs 10Mb/s

But the NFS access to the memory drive is still 5MB/s :-(

As there is no more hard drive involved, we know that there is a 
bottleneck at 5MB in NFS layer on this system... Where ?

Thanks a lot for any help on the method to find/diagnose this !

----------------------------------------Details are below :

nfsserver# uname -a
FreeBSD nfsserver 7.1-RC1 FreeBSD 7.1-RC1 #0: Sun Dec  7 00:38:13 UTC 
2008     root at driscoll.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC  amd64
nfsserver# mdconfig -a -t swap -s 600m -o reserve -u 7
nfsserver# ls /dev/md*
/dev/md7        /dev/mdctl
nfsserver# newfs -i 2048 /dev/md7
Reduced frags per cylinder group from 60864 to 59928 to enlarge last cyl 
group
/dev/md7: 600.0MB (1228800 sectors) block size 16384, fragment size 2048
         using 6 cylinder groups of 117.05MB, 7491 blks, 59968 inodes.
super-block backups (for fsck -b #) at:
  160, 239872, 479584, 719296, 959008, 1198720
nfsserver# mkdir /tstnfs
nfsserver# mount /dev/md7 /tstnfs

nfsserver# date ; time tar -cf - 
/nfsro/commun/clientusr-amd64-7.2-RC2-20081230/ports > /dev/null ; date ;
Wed Dec 31 09:11:08 CET 2008
tar: Removing leading '/' from member names
3.794u 8.766s 0:46.40 27.0%     71+1406k 123375+0io 0pf+0w
Wed Dec 31 09:11:54 CET 2008

That makes 498MB / 46s = 10.8MB/s for disk drive.

nfsserver# date ; cp -r -p 
/nfsro/commun/clientusr-amd64-7.2-RC2-20081230/ports /tstnfs/ports ; date
Wed Dec 31 09:33:09 CET 2008
Wed Dec 31 09:34:46 CET 2008

df -h
/dev/md7       512M    498M    -27M   106%    /tstnfs

nfsserver# date ; time tar -cf - /tstnfs/ports > /dev/null ; date ;
Wed Dec 31 09:36:59 CET 2008
tar: Removing leading '/' from member names
2.947u 6.218s 0:10.61 86.2%     74+1463k 104885+0io 0pf+0w
Wed Dec 31 09:37:10 CET 2008
nfsserver# date ; time tar -cf - /tstnfs/ports > /dev/null ; date ;
Wed Dec 31 09:37:12 CET 2008
tar: Removing leading '/' from member names
2.895u 6.487s 0:11.01 85.1%     74+1466k 112838+0io 0pf+0w
Wed Dec 31 09:37:23 CET 2008
nfsserver# date ; time tar -cf - /tstnfs/ports > /dev/null ; date ;
Wed Dec 31 09:40:22 CET 2008
tar: Removing leading '/' from member names
2.902u 6.610s 0:11.10 85.6%     75+1483k 113393+0io 0pf+0w
Wed Dec 31 09:40:33 CET 2008

That makes 498MB / 11s = 45MB/s : better that 10MB/s for disk, but not 
exceptional.

Now on the client :

client9# uname -a
FreeBSD client9 7.1-RC2 FreeBSD 7.1-RC2 #0: Tue Dec 23 11:42:13 UTC 2008 
     root at driscoll.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC  amd64

client9# mount -o ro nfsserver:/tstnfs /tstnfs
client9# df -h
nfsserver:/tstnfs                                         512M    498M 
     0B   100%    /tstnfs

client9# date ; time tar -cf - /tstnfs/ports > /dev/null ; date ;
Wed Dec 31 09:50:39 CET 2008
tar: Removing leading '/' from member names
2.896u 13.020s 1:35.22 16.7%    75+1483k 0+0io 2pf+0w
Wed Dec 31 09:52:14 CET 2008
client9# date ; time tar -cf - /tstnfs/ports > /dev/null ; date ;
Wed Dec 31 09:52:22 CET 2008
tar: Removing leading '/' from member names
2.700u 12.755s 1:27.78 17.6%    76+1498k 0+0io 0pf+0w
Wed Dec 31 09:53:50 CET 2008
client9# date ; time tar -cf - /tstnfs/ports > /dev/null ; date ;
Wed Dec 31 09:55:02 CET 2008
tar: Removing leading '/' from member names
2.681u 12.688s 1:28.15 17.4%    74+1464k 0+0io 0pf+0w
Wed Dec 31 09:56:30 CET 2008

That makes between 95s and 87s, then 498MB / 95s = 5,2MB/s and 5.7MB/s, 
like previous test from hard drive NFS export.


Top is showing around 100MB of free memory while taring on client9, so i 
don't think tar is paging on network :
last pid:  3318;  load averages:  0.17,  0.11,  0.04    up 0+11:14:27 
10:08:10
30 processes:  1 running, 29 sleeping
CPU:  0.8% user,  0.0% nice,  9.0% system,  0.0% interrupt, 90.2% idle
Mem: 19M Active, 720M Inact, 136M Wired, 240K Cache, 110M Buf, 98M Free



Best regards,
-- 
Bernard DUGAS Mobile +33 615 333 770


More information about the freebsd-questions mailing list