Reading via mmap stinks (Re: weird bugs with mmap-ing via NFS)
peterjeremy at optushome.com.au
Sat Mar 25 21:27:40 UTC 2006
On Sat, 2006-Mar-25 09:20:13 -0500, Mikhail Teterin wrote:
>I'm sorry, that should be http://aldan.algebra.com/~mi/mzip.c -- I checked
>this time :-(
It doesn't look like it's doing anything especially weird. As Matt
pointed out, creating files with mmap() is not a good idea because the
syncer can cause massive fragmentation when allocating space. I can't
test is as-is because it insists on mmap'ing its output and I only
have one disk and you can't mmap /dev/null.
Since your program is already written to mmap the input and output in
pieces, it would be trivial to convert it to use read/write.
>= I tried writing a program that just mmap'd my entire (2GB) test file
>= and summed all the longwords in it.
>The files I'm dealing with are database dumps -- 10-80Gb :-) Maybe, that's,
>what triggers some pessimal case?..
I tried generating an 11GB test file and got results consistent with my
previous tests: grep using read or mmap, as well as mmap'ing the entire
file give similar times with the disk mostly saturated.
I suggest you try converting mzip.c to use read/write and see if the
problem is still present.
More information about the freebsd-stable