Linux NFS ate my bge
Matthew Dillon
dillon at apollo.backplane.com
Wed Jul 22 23:07:38 UTC 2009
:requests.
:
:I've (re)discovered this now that I've recently got my assorted config
:problems largely ironed out and have my FreeBSD box up to date again, I'm
:reminded of an unresolved problem from way back, which is that my bge card
:collapses after being subjected to a large amount of NFS traffic coming
:from my Linux box, e.g. recompiling KDE on a discless workstation, which
:has been responsible for three embuggerances so far today.
If you are using a NFS UDP mount, try using a NFS TCP mount instead.
This could very well instantly fix your issues even if it does not
solve the underlying bugs.
With a UDP mount the server can get a barrage of IP fragmented UDP
packets, particularly from linux clients. While I don't know any
specific with regards to bugs in the bge driver itself, I do know that
for a UDP mount to operate adequately over a GigE network the NFS server
needs about a 2 Megabyte socket buffer to receive the bursts.
This is not something that would typically be seen between two FreeBSD
boxes as FreeBSD's asynchronous NFS client traffic (mainly read-ahead)
is limited by available synchronous nfsiod threads, and further limited
by default mount options, internal #defined concurrency limits, and
by the out-of-order transmission of the read ahead rpcs caused by the
way the nfsiod threads operate (which results in out-of-order processing
on the server side and stalls the linear reads done by the client).
Linux clients, on the other-hand can generate an enormous number of
concurrent RPCs.
Use of a TCP mount instead of a UDP mount solves the sockbuf and
IP fragmentation issues. The TCP connection will not use fragmented IP
packets, will not blow away the server's receive-side sockbuf, and
does a much better job dealing with any packet loss, to boot.
-Matt
More information about the freebsd-current
mailing list