Linux NFS ate my bge

Matthew Dillon dillon at apollo.backplane.com
Wed Jul 22 23:07:38 UTC 2009


:requests.
:
:I've (re)discovered this now that I've recently got my assorted config 
:problems largely ironed out and have my FreeBSD box up to date again, I'm 
:reminded of an unresolved problem from way back, which is that my bge card 
:collapses after being subjected to a large amount of NFS traffic coming 
:from my Linux box, e.g. recompiling KDE on a discless workstation, which 
:has been responsible for three embuggerances so far today.

    If you are using a NFS UDP mount, try using a NFS TCP mount instead.
    This could very well instantly fix your issues even if it does not
    solve the underlying bugs.

    With a UDP mount the server can get a barrage of IP fragmented UDP
    packets, particularly from linux clients.  While I don't know any
    specific with regards to bugs in the bge driver itself, I do know that
    for a UDP mount to operate adequately over a GigE network the NFS server
    needs about a 2 Megabyte socket buffer to receive the bursts.

    This is not something that would typically be seen between two FreeBSD
    boxes as FreeBSD's asynchronous NFS client traffic (mainly read-ahead)
    is limited by available synchronous nfsiod threads, and further limited
    by default mount options, internal #defined concurrency limits, and
    by the out-of-order transmission of the read ahead rpcs caused by the
    way the nfsiod threads operate (which results in out-of-order processing
    on the server side and stalls the linear reads done by the client).

    Linux clients, on the other-hand can generate an enormous number of
    concurrent RPCs.

    Use of a TCP mount instead of a UDP mount solves the sockbuf and
    IP fragmentation issues.  The TCP connection will not use fragmented IP
    packets, will not blow away the server's receive-side sockbuf, and
    does a much better job dealing with any packet loss, to boot.

						-Matt



More information about the freebsd-current mailing list