Reading via mmap stinks (Re: weird bugs with mmap-ing via NFS)
mi+kde at aldan.algebra.com
Sun Mar 26 16:45:15 UTC 2006
On Saturday 25 March 2006 06:46 pm, Peter Jeremy wrote:
= My guess is that the read-ahead algorithms are working but aren't doing
= enough re-ahead to cope with "read a bit, do some cpu-intensive processing
= and repeat" at 25MB/sec so you're winding up with a degree of serialisation
= where the I/O and compressing aren't overlapped. I'm not sure how tunable
= the read-ahead is.
Well, is the MADV_SEQUNTIAL advice, given over the entire mmap-ed region,
taken into account anywhere in the kernel? The kernel could read-ahead more
aggressively if it freed the just accessed pages faster, than it does in the
Matt wrote in the same thread:
= It is particularly possible when you combine read() with
= mmap because read() uses a different heuristic then mmap() to
= implement the read-ahead. There is also code in there which depresses
= the page priority of 'old' already-read pages in the sequential case.
Well, thanks for the theoretical confirmation of what I was trying to prove by
experiments :-) Can this depressing of the "old" pages in the sequential
case, that read's implementation already has, be also implemented in mmap's
case? It may not *always* be, what the mmap-ing program wants, but when the
said program uses MADV_SEQUENTAIL, it should not be ignored... (Bakul
understood this point of mine 3 days ago :-)
Peter Jeremy also wrote, in another message:
= I can't test is as-is because it insists on mmap'ing its output and I only
= have one disk and you can't mmap /dev/null.
If you use a well compressible (redundant) file, such as a web-server log, and
a high enough compression ratio, you can use the same disk for output -- the
writes will be very infrequent.
More information about the freebsd-stable