nfs and ipfw

[Crist J. Clark]

On Sun, Apr 27, 2003 at 08:08:11PM -0500, Robert Johannes wrote:
[snip]

> I'm using normal ipfw, with the following rules:
> 
> allow ip from any to any via lo0
> deny ip from any to 127.0.0.0/8
> deny ip from 127.0.0.0/8 to any
> allow tcp from any to any established
> allow ip from any to any frag
> allow tcp from any to any setup
> allow ip from $nfsclient to $fileserver keep-state
> allow ip from xx.xx.xx.1 to $fileserver keep-state
> deny ip from any to any
> 
> 
> The router/gateway is at xx.xx.xx.254.  I'm able to mount the filesystems
> from the $fileserver, but I'm not able to write a substantial amount of
> data to the filesystems; I can create a file by 'touching' one on the nfs
> filesyste, but I can't copy a big file onto the filesystem.  I have
> successfully copied a file as big as the /etc/hosts files (a few bytes).
> >From watching tcpdump, it seems that any time there's significant i/o on
> the nfs filesystem, the fileserver stops responding, and I note the
> following lines repeated perhaps a hundred or more times:
> 
> 15:04:32.619887 $nfsclient > $nfsserver: (frag 7506:340 at 32560)
> 15:04:32.619906 $nfsclient > $nfsserver: (frag 7506:1480 at 31080+)
> 15:04:32.619934 $nfsclient > $nfsserver: (frag 7506:1480 at 29600+)
> 15:04:32.619949 $nfsclient > $nfsserver: (frag 7506:1480 at 28120+)
> 15:04:32.619962 $nfsclient > $nfsserver: (frag 7506:1480 at 26640+)
> 15:04:32.619975 $nfsclient > $nfsserver: (frag 7506:1480 at 25160+)
> 15:04:32.619987 $nfsclient > $nfsserver: (frag 7506:1480 at 23680+)
> 15:04:32.619998 $nfsclient > $nfsserver: (frag 7506:1480 at 22200+)
> 15:04:32.620009 $nfsclient > $nfsserver: (frag 7506:1480 at 20720+)
> 
> At this point I get an "nfs: server $nfsserver not responding, timed out"
> message logged on the nfsclient.
> 
> I'm pretty sure it has to do with my ipfw configuration, but I can't
> pinpoint the problem.  Any ideas?

It looks like those fragments should be passing the 'frag' rule. Check
if those fragments are really being dropped. Turn on logging in the
last 'deny' rule to see for sure. If that's not it, the log might give
you a clue as to what the problem really is anyway.

The possible way around this is to do NFS over TCP which won't
generate the hella-huge UDP packets.
-- 
Crist J. Clark                     |     cjclark at alum.mit.edu
                                   |     cjclark at jhu.edu
http://people.freebsd.org/~cjc/    |     cjc at freebsd.org