scp more perfectly fills the pipe than NFS/TCP

Matthew Dillon dillon at apollo.backplane.com
Mon Dec 21 23:05:30 UTC 2009


    Oh, one more thing... I'm assuming you haven't used tcpdump with
    NFS much.  tcpdump has issues parsing the NFS RPC's out of a TCP
    stream.  For the purposes of testing you may want to temporarily
    use a UDP NFS mount.  tcpdump can parse the NFS RPCs out of the UDP
    stream far more easily.  If you use a UDP mount use the dumbtimer
    option and set it to something big, like 10 seconds, so you don't
    get caught up in NFS/UDP's retry code (which will confuse your
    parsing of the output).

    A typical tcpdump line would be something like this:

    tcpdump -n -i nfe0 -l -s 4096 -vvv not port 2049

    Where the port is whatever port the NFS RPC's are running over
    while you are running the test.  You'd want to display it on a
    max-width xterm, or record a bunch of it to a file and then review
    it via less.

    The purpose of running the tcpdump is to validate all your assumptions
    as well as determine whether basic features such as read-ahead are
    actually running.  You can also determine if packet loss is occuring,
    if requests are being sent or responded to out of order (the RPC
    tcpdump parses includes the request id's and the file offsets so it
    should be easy to figure that out).  You can also determine the
    actual latency by looking at the timestamps for the request vs
    the reply.

    Once you've figured out as much as you can from that you can try
    tcpdumping the TCP stream.  In this case you may not be able to
    pick out RPCs but you should be able to determine whether the
    requests are being pipelined and whether any packet loss is occurring
    or not.  You can also determine whether the TCP link is working
    properly... i.e. that the TCP packets are properly flagging the
    'P'ushes and not delaying the responses, and that the link isn't
    blowing out its socket buffer or TCP window (those are two separate
    things).  The kernel should be scaling things properly but you never
    know.

						-Matt



More information about the freebsd-hackers mailing list