Struggling on NFS problem

[Cheng-Lin Yang]

Hi Erich,
Here is the detail information:
1. The RAM on machine is: hw.physmem: 4278906880 

2. My RAIDZ config
$ zpool status csdata
  pool: csdata
 state: ONLINE
 scrub: none requested
config:

	NAME        STATE     READ WRITE CKSUM
	csdata      ONLINE       0     0     0
	  da0       ONLINE       0     0     0

errors: No known data errors

3. I do not have a separate ZIL device like a solid state drive

4. uname -a
FreeBSD cs1.cs.ccu.edu.tw 8.0-RELEASE FreeBSD 8.0-RELEASE #0: Sat Nov 21 15:02:08 UTC 2009     root at mason.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC  amd64

Also, the detail dmesg output can be found here: http://www.cs.ccu.edu.tw/~hsp97m/dmesg.txt

I'm looking forward to have your suggestions. Thank you :)

Regards,
Cheng-Lin Yang

-----Original message-----
From:Erich Jenkins, Fuujin Group Ltd <erich at fuujingroup.com>
To:yuwen at exodus.cs.ccu.edu.tw
Cc:freebsd-fs <freebsd-fs at freebsd.org>,lab <lab at cs.ccu.edu.tw>
Date:Tue, 04 May 2010 03:26:27 -0600
Subject:Re: Struggling on NFS problem

Cheng-Lin Yang wrote:
> Dear all,
> Currently, we have a NFS server which runs FreeBSD8 with ZFS and few workstations as NFS client (2 * FreeBSD8 amd64 + 1 * FreeBSD7.2 i386 + 2 * Fedora + Debian). We spotted that NFS performs weirdly on FreeBSD clients, which will significantly slow down the system response. The only solution to it is to reboot the clients (Linux client runs smoothly). So we try to use "nfsstat -c" on FreeBSD client to dig into the problem and found strange result (http://pastebin.com/K71qpEDG) :
> csie0[~]# nfsstat -c
> Client Info:
> Rpc Counts:
>   Getattr   Setattr    Lookup  Readlink      Read     Write    Create    Remove
>   3595047     75100   1089213      1095   1258311  48664374    104346    116376
>    Rename      Link   Symlink     Mkdir     Rmdir   Readdir  RdirPlus    Access
>     17621      1644        77     19569      9459     45177         0    323665
>     Mknod    Fsstat    Fsinfo  PathConf    Commit
>         0    111262         3         0    590876
> Rpc Info:
>  TimedOut   Invalid X Replies   Retries  Requests
>         0         0         0         0  56022746
> Cache Info:
> Attr Hits    Misses Lkup Hits    Misses BioR Hits    Misses BioW Hits    Misses
> 198414412   3595052 112100649   1088903  62793893   1253320  -1008715  48664379
> BioRLHits    Misses BioD Hits    Misses DirE Hits    Misses
>      3751      1095    204326     45191    109194         8
> 
> As you can see, the value of "BioW Hits" is a negative number. Shouldn't it be equal or larger than zero? We have totally no idea on this issue. Please kindly help us on investigating the problem. Any suggestion is extremely welcomed. Thank you.
> 
> Regards,
> George
> _______________________________________________
> freebsd-fs at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-fs
> To unsubscribe, send any mail to "freebsd-fs-unsubscribe at freebsd.org"

Cheng-Lin

Could you provide a bit more info about your system and setup? How much
RAM, what type of raidz config? Do you have a separate ZIL device like a
solid state drive? Often, you'll experience severe NFS performance
degradation when using raidz and nfs if the zil logs are on the same
drives as the raidz volume.


Erich M. Jenkins
Fuujin Group Limited

"You should never, never doubt what no one is sure about."
-- Gene Wilder



----------------------------------
Cheng-Lin Yang
Sun Certified Java Programmer
High Speed Network Group Lab (HSNG)
Institute of Computer Science & Info. Engineering,
National Chung Cheng Univerisity, Taiwan
E-mail: yuwen at exodus.cs.ccu.edu.tw