rpcsvc wcpu 500%

[Gil Vidals]

My FreeBSD 8.1 running 64-bit / 8 GB of RAM from two mirrored HP USB Sticks.
The FreeBSD 8.1 server is a new NAS for my ESX Hosting environment. The file
system I am using is ZFS and the ESX hosts connect as clients to the NAS. I
am using link aggregation on two NICs. The FreeBSD NAS hung when WCPU went
to 500 percent. It happened after I started a second mount point from my ESX
Host to the NFS store. Since I read a thread from another soul who
experience the similar situation when launching VMs on a SECOND mount point
(NFS), the assumption was that FreeBSD 8.1 doesn't run more than one NFS
Server and that is what causes the crash, but I've been corrected by others
in this forum. FreeBSD 8.1 runs one NFS Master and Server, but many threads.
I do now see that with the ps wauxH | grep nfs command.

In any case, I'm back to square one. How should I go about investigating
what caused the RPCSVC spike to 500% causing my VMs to hang and I had to
hard reboot the server?  (i tried shutting down nfsd, but that just hung).


last pid:  3899;  load averages:  0.00,  0.00,
0.00
up 0+15:58:33  09:21:15
62 processes:  1 running, 61 sleeping
CPU:  0.0% user,  0.0% nice,  0.0% system,  0.0% interrupt,  100% idle
Mem: 18M Active, 11M Inact, 2163M Wired, 4K Cache, 557M Buf, 5719M Free
Swap: 3942M Total, 3942M Free

  PID USERNAME  THR PRI NICE   SIZE    RES STATE   C   TIME   WCPU COMMAND
 1214 root       24  44    0  5804K  1588K rpcsvc  2   8:38  0.00% nfsd
<======= WCPU had spiked to 500%


zambia#  ps wauxH | grep nfs
root     1213  0.0  0.0  5804  1508  ??  Is    5:23PM   0:00.03 nfsd: master
(nfsd)
root     1214  0.0  0.0  5804  1588  ??  S     5:23PM   0:21.79 nfsd: server
(nfsd)
root     1214  0.0  0.0  5804  1588  ??  S     5:23PM   0:22.25 nfsd: server
(nfsd)
root     1214  0.0  0.0  5804  1588  ??  S     5:23PM   0:21.02 nfsd: server
(nfsd)
root     1214  0.0  0.0  5804  1588  ??  S     5:23PM   0:21.17 nfsd: server
(nfsd)
root     1214  0.0  0.0  5804  1588  ??  S     5:23PM   0:21.77 nfsd: server
(nfsd)
root     1214  0.0  0.0  5804  1588  ??  S     5:23PM   0:21.31 nfsd: server
(nfsd)
root     1214  0.0  0.0  5804  1588  ??  S     5:23PM   0:20.15 nfsd: server
(nfsd)
root     1214  0.0  0.0  5804  1588  ??  S     5:23PM   0:21.99 nfsd: server
(nfsd)
root     1214  0.0  0.0  5804  1588  ??  S     5:23PM   0:21.86 nfsd: server
(nfsd)
root     1214  0.0  0.0  5804  1588  ??  S     5:23PM   0:23.37 nfsd: server
(nfsd)
root     1214  0.0  0.0  5804  1588  ??  S     5:23PM   0:22.36 nfsd: server
(nfsd)
root     1214  0.0  0.0  5804  1588  ??  S     5:23PM   0:22.57 nfsd: server
(nfsd)
root     1214  0.0  0.0  5804  1588  ??  S     5:23PM   0:20.13 nfsd: server
(nfsd)
root     1214  0.0  0.0  5804  1588  ??  S     5:23PM   0:22.62 nfsd: server
(nfsd)
root     1214  0.0  0.0  5804  1588  ??  S     5:23PM   0:21.30 nfsd: server
(nfsd)
root     1214  0.0  0.0  5804  1588  ??  S     5:23PM   0:22.47 nfsd: server
(nfsd)
root     1214  0.0  0.0  5804  1588  ??  S     5:23PM   0:20.99 nfsd: server
(nfsd)
root     1214  0.0  0.0  5804  1588  ??  S     5:23PM   0:22.24 nfsd: server
(nfsd)
root     1214  0.0  0.0  5804  1588  ??  S     5:23PM   0:21.26 nfsd: server
(nfsd)
root     1214  0.0  0.0  5804  1588  ??  S     5:23PM   0:22.09 nfsd: server
(nfsd)
root     1214  0.0  0.0  5804  1588  ??  S     5:23PM   0:21.72 nfsd: server
(nfsd)
root     1214  0.0  0.0  5804  1588  ??  S     5:23PM   0:20.60 nfsd: server
(nfsd)
root     1214  0.0  0.0  5804  1588  ??  S     5:23PM   0:21.07 nfsd: server
(nfsd)
root     1214  0.0  0.0  5804  1588  ??  S     5:23PM   0:21.03 nfsd: server
(nfsd)

Thanks,
Gil Vidals