Odd network issues on ZFS based NFS server

[Anders Nordby]

Hi,

On Wed, Jun 09, 2010 at 11:28:52AM -0400, Rick Macklem wrote:
> When you tried a different NIC, was a different type (ie. different
> chipset that uses a different device driver)? I suggested that not
> because I thought the hardware was broken but because I thought it
> might be related to the network interface's device driver and switching
> to a different device driver would isolate that possibility.

Nope. I switched from NIC 1 to 2, and switched server to an identical
one. They both use bge NICs, a very common interface. I somehow doubt
this is related to the NIC or driver, I have many machines with the same
bge NIC (HP NC7782) that does not have any problems like this.

> Well, it doesn't seem to be mbuf exhaustion (I don't know what
> "out of packet secondary zone" means, I'll have to look at that) and
> if it doesn't handle pings it seems really hosed. Have you done a
> "vmstat 5" + "ps axlH" (or similar) to try and see what it's doing?
> ("top" and "netstat" might also help?)

root at unixfile:~# vmstat 5
 procs      memory      page                    disks     faults
cpu
 r b w     avm    fre   flt  re  pi  po    fr  sr da0 da1   in   sy   cs
us sy id
 0 0 0    410M  1781M   279   0   0   0   481   0   0   0 1918 12338
6476  0  2 98
 0 0 0    410M  1781M     1   0   0   0     0   0   0   0  497   34 2268
0  1 99
 0 0 0    410M  1781M   123   0   0   0   116   0   0   0  455 1787 2071
0  0 99
 0 0 0    410M  1781M     0   0   0   0     4   0   0   0  292   38 1459
0  1 99
^C
root at unixfile:~# top -b 5
last pid: 86306;  load averages:  0.04,  0.13,  0.07  up 0+22:01:28
13:09:31
46 processes:  1 running, 45 sleeping

Mem: 25M Active, 147M Inact, 1021M Wired, 112K Cache, 316M Buf, 1780M
Free
Swap: 6144M Total, 6144M Free


  PID USERNAME  THR PRI NICE   SIZE    RES STATE   C   TIME   WCPU
COMMAND
  786 root        4  44    0  5804K  1276K rpcsvc  1  51:04  0.00% nfsd
  839 nagios      1  44    0 10880K  3228K select  0   0:04  0.00% nrpe2
  847 root        1  44    0 20852K  8356K select  0   0:04  0.00%
perl5.10.1
 1076 root        1  44    0 11968K  4188K select  0   0:01  0.00%
sendmail
81645 root        1  44    0 10220K  2920K wait    2   0:01  0.00% bash

The server doesn't have many connections, 16 in ESTABLISHED state. As
you can see from top, the server has 1780 MB free memory.

Regards,

-- 
Anders.