Reading via mmap stinks (Re: weird bugs with mmap-ing via NFS)

[Peter Jeremy]

On Sat, 2006-Mar-25 09:20:13 -0500, Mikhail Teterin wrote:
>I'm sorry, that should be  http://aldan.algebra.com/~mi/mzip.c -- I checked 
>this time :-(

It doesn't look like it's doing anything especially weird.  As Matt
pointed out, creating files with mmap() is not a good idea because the
syncer can cause massive fragmentation when allocating space.  I can't
test is as-is because it insists on mmap'ing its output and I only
have one disk and you can't mmap /dev/null.

Since your program is already written to mmap the input and output in
pieces, it would be trivial to convert it to use read/write.

>= I tried writing a program that just mmap'd my entire (2GB) test file
>= and summed all the longwords in it.
>
>The files I'm dealing with are database dumps -- 10-80Gb :-) Maybe, that's,
>what triggers some pessimal case?..

I tried generating an 11GB test file and got results consistent with my
previous tests:  grep using read or mmap, as well as mmap'ing the entire
file give similar times with the disk mostly saturated.

I suggest you try converting mzip.c to use read/write and see if the
problem is still present.

-- 
Peter Jeremy