[SOLVED] Re: NFS UDP mounts on RELENG_6?

[Jon Dama]

No shame.  This is a common problem, and if you think real hard you should
start getting quasy about UDP NFS under high load.  The problem arises
because if part of a UDP packet is lost the entire packet is lost, and as
UDP NFS uses fixed packet sizes... this means the system never recovers if
frames stop droping perpetually.  Its easy to see how this happens for
speed mismatches, the switch (vs a router) provides no buffering and
therefore frames from the fast source are dropped.

Now imagine that you have many clients all writing data to the NFS server
at once... Suddently that pipe into the server is like a narrow straw.
Ooops.

I haven't see any evidence that suggests using NFS with UDP is actually
useful.  IMO, its a false economy.  TCP processing takes on the order of
1uS of CPU time--which is on the order of the frame latency through a
single switch!  That is to say, nothing, but the behavior of TCP NFS under
load (when it counts) is superior.

TCP SACK and interrupt aggregation are better ways of squeezing extra
performance out of your hardware than simply using UDP...

Just my two cents.

-Jon

On Mon, 19 Dec 2005, Oliver Brandmueller wrote:

> Hi.
>
> On Mon, Dec 19, 2005 at 01:01:34AM -0800, Jon Dama wrote:
> > A very critical question here is the network topology.
> >
> > UDP NFS _cannot_ be used across switches where the ports are operating at
> > different speeds--unless the UDP packet size is to be smaller than MTU.
> >
> > Be sure and verify that every link between the server and the client are
> > operating at the same speed.
>
> *ouch* shame on me.
>
> I looked at interfaces, links, errors - everything. I found the problem
> in a misconfiguration and you just pointed at it:
>
> The server has not been booted for a few hundred days before upgrading.
> >From an old test there was an "mtu 9000" for the NFS interface still in
> /etc/rc.conf (while it has been reset after failed tests with other
> hardware on that network manually to 1500).
>
> So: SORRY for making everybody mad here. It was just me being blind.
>
> Thanx for pointing me at that!
>
> - Oliver
>
> --
> | Oliver Brandmueller | Offenbacher Str. 1  | Germany       D-14197 Berlin |
> | Fon +49-172-3130856 | Fax +49-172-3145027 | WWW:   http://the.addict.de/ |
> |               Ich bin das Internet. Sowahr ich Gott helfe.               |
> | Eine gewerbliche Nutzung aller enthaltenen Adressen ist nicht gestattet! |
>