more weird bugs with mmap-ing via NFS

[Oliver Fromme]

Matthew Dillon <dillon at apollo.backplane.com> wrote:
 >     There are a number of problems using a block size of 65536.  First of
 >     all, I think you can only safely do it if you use a TCP mount, also
 >     assuming the TCP buffer size is appropriately large to hold an entire
 >     packet.  For UDP mounts, 65536 is too large (the UDP data length can
 >     only be 65536 bytes.  For that matter, the *IP* packet itself can 
 >     not exceed 65535 bytes.  So 65536 will not work with a UDP mount.
 > 
 >     The second problem is related to the network driver.  The packet MTU
 >     is 1500, which means, typically, a limit of around 1460-1480 payload
 >     bytes per packet.  A UDP large UDP packet that is, say, 48KB, will be
 >     broken down into over 33 IP packet fragments.  The network stack could
 >     very well drop some of these packet fragments making delivery of the 
 >     overall UDP packet unreliable.

In fact, FreeBSD has a default limit of 16 fragments.  That
means that the maximum packet size is about 25 Kbyte (with
the default MTU of 1500).

$ sysctl net.inet.ip.maxfragsperpacket
net.inet.ip.maxfragsperpacket: 16

I would advise against increasing that limit.  First, you
would have to increase it on _all_ machines you communicate
with.  Seconds, it increases the reassembly overhead in the
kernel considerably, and it make the situation much worse
if there's the slightest chance of packet drop.  It will
also make you more vulnerable for certain kinds of DoS
attacks.

I think Matt's recommendations are right:  Don't fiddle
with the block sizes at all.  Use the defaults, which have
been carefully chosen.  Use TCP instead of UDP.  Don't use
mmap() fro writing large sequential files.  write() is much
better.

Best regards
   Oliver

PS:  Excerpt from the mount_nfs(8) manpage:
"Due to the way that Sun RPC is implemented on top of
UDP (unreliable datagram) transport, tuning such mounts
is really a black art that can only be expected to have
limited success."

-- 
Oliver Fromme,  secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing
Dienstleistungen mit Schwerpunkt FreeBSD: http://www.secnetix.de/bsd
Any opinions expressed in this message may be personal to the author
and may not necessarily reflect the opinions of secnetix in any way.

"I invented Ctrl-Alt-Delete, but Bill Gates made it famous."
        -- David Bradley, original IBM PC design team