more weird bugs with mmap-ing via NFS

[Mikhail Teterin]

вівторок 21 березень 2006 19:25, Matthew Dillon Ви написали:
>     If the program works over a local
>     filesystem but fails to produce data in the output file on an NFS
>     mount (but completes otherwise), then there is a bug in NFS somewhere.
>     If the problem is simply due to the program stalling, and not
>     completing due to the stalling, then it could be a problem with dropped
>     packets in the network stack.

The file stops growing, but the network bandwidth remains at 20Mb/s. `Netstat 
-s' on the client, had the following to say (udp and ip only):

	udp:
	        20417616 datagrams received
	        0 with incomplete header
        	0 with bad data length field
	        0 with bad checksum
	        4 with no checksum
	        14 dropped due to no socket
	        89 broadcast/multicast datagrams dropped due to no socket
	        0 dropped due to full socket buffers
	        0 not for hashed pcb
	        20417513 delivered
	        20409284 datagrams output

and
	ip:
        	45653354 total packets received
	        0 bad header checksums
	        0 with size smaller than minimum
	        0 with data size < data length
	        0 with ip length > max ip packet size
	        0 with header length < data size
	        0 with data length < header length
	        0 with bad options
	        0 with incorrect version number
	        210876 fragments received
	        0 fragments dropped (dup or out of space)
	        0 fragments dropped after timeout
	        32712 packets reassembled ok
	        45474799 packets for this host
	        135 packets for unknown/unsupported protocol
        	0 packets forwarded (0 packets fast forwarded)
	        256 packets not forwardable
	        0 packets received for unknown multicast group
	        0 redirects sent
	        43338317 packets sent from this host
	        0 packets sent with fabricated ip header
	        0 output packets dropped due to no bufs, etc.
        	0 output packets discarded due to no route
	        18604565 output datagrams fragmented
	        164573586 fragments created
        	0 datagrams that can't be fragmented
	        0 tunneling packets that can't find gif
	        0 datagrams with bad address in header

Nothing drastically alarming on the server either. Is this bad:
	ip:
		[...]
	        3241 fragments dropped (dup or out of space)

I don't think so -- the client was writing at 20Mb/s since the beginning of 
this thread...

>     If the problem is that the program simply runs very inefficiently over
>     NFS, with excessive network bandwidth for the data being written (as you
>     also reported), this is probably an artifact of attempting to use mmap()
>     to write out the data, for reasons previously discussed.

The reasons previously discussed could excuse the difference between actual 
and expected performance of no more than an order of (decimal) magnitude. I'm 
seeing actual bandwidth use of 20Mb/second. Even if the useful part of that 
were mere 1% (TWO orders of magnitude), the file would be growing at about 
200Kb/s. It is not...

The stalling caused by `systat -vm' can not be excused by mmap ineficiencies 
at all, can it?

>    I would again caution against using mmap() to populate a file in this
>    manner.  Even with MADV_SEQUENTIAL there is no guarentee that the system
>    will actually flush the pages to the actual file on the server
>    sequentially, and you could end up with a very badly fragmented file.
>    [...] The VM system does its best, but

I'd argue, that this does not seem like "best" at all, but I, actually, do not 
care in this example -- the backups are never read again in normal 
circumstances. In the infrequent abnormal ones, they will be uncompressed 
first (a CPU- rather than I/O-intensive process) and only then used for 
recovery.

Yours,

	-mi