Odd network issues on ZFS based NFS server

[Pyun YongHyeon]

On Fri, Jun 11, 2010 at 06:33:14PM +0200, Anders Nordby wrote:
> Hi,
> 
> On Thu, Jun 10, 2010 at 08:18:09PM -0700, Jeremy Chadwick wrote:
> >> Given that you stated FreeBSD8.1-Prerelease I think you should have the
> >> patch, but please make sure that your sys/nfsserver/nfs_srvsubs.c is
> >> at least r206406.
> 
> I didn't have any time to dump and look at the network traffic much yet
> (life is busy). But, the issue in this thread also happens/happened in
> FreeBSD 7.3-RELEASE, so I don't see how it's a recent change that makes
> this happen. Last night I had some progress, by switching to an old 100
> Mbps USB NIC of mine (nerds sure do have lots of handy things at home
> eh) I got rid of the packet loss:
> 
> Jun 11 01:25:14 unixfile kernel: rue0: <USBKR100 USB 10/100 LAN, class
> 0/0, rev 
> 1.10/1.00, addr 2> on usbus3
> Jun 11 01:25:14 unixfile kernel: miibus2: <MII bus> on rue0
> Jun 11 01:25:14 unixfile kernel: ruephy0: <RealTek RTL8150 internal
> media interf
> ace> PHY 0 on miibus2
> 
> Performance is quite lousy however. Just in case I am trying to get hold
> of a PCI-X Intel NIC to see how that goes, as this is a production
> server after all (or supposed to be).
> 
> > With regards to possible bge(4) issues, Yong-Hyeon works on this driver
> > fairly often.  If it turns out to be a driver issue of some sort, he can
> > probably help.  Relevant commits are here (to give you some idea of
> > activity):
> > 
> > http://www.freebsd.org/cgi/cvsweb.cgi/src/sys/dev/bge/if_bge.c
> > 
> > One commit caught my eye (rev 1.226.2.15), but that seems to be more
> > focused on mbuf issues (your system doesn't appear to be having any,
> > given your netstat -m output).
> > 
> > CC'ing Yong-Hyeong, as he might know of some edge case where bge(4)
> > could go crazy with interrupts.  :-)  Yong-Hyeon, the entire thread is
> > here:
> > 
> > http://lists.freebsd.org/pipermail/freebsd-fs/2010-June/008654.html
> 
> Let me know if there's anything bge related I can try/test. It might
> take a day or two or more. Customer is sort of getting annoyed by these
> problems, so the room for testing is getting smaller. But of course I
> want to help get a fix for this.
> 

Show me dmesg output to know which bge(4) controller you had. And
show me output of "netstat -ndI bge0". Some bge(4) controllers
supports detailed MAC counters and these are exported via sysctl.
If your controller is one of these controller, you can check the
statistics of controller with "sysctl dev.bge.0.stat" and post it
if you can see them.

> Regards,
> 
> -- 
> Anders.