How does disk caching work?

[Jim C. Nasby]

Thanks very much for all your help. Only remaining question I have is:
is there something we can monitor to gauge what impact (if any) changes
to these settings have? Will changes to hi|lowrunningspace just result
in increased KB/s or TPS write performance in gstat? How can we tell if
we've advanced it too much?

Likewise, what can we measure regarding nbuf? If I'm understanding
things correctly, runningspace comes out of nbuf, so obviously it needs
to be greater than that, but what symptoms will we see if it's set too
low?

On Mon, Apr 19, 2004 at 06:25:47PM +0200, Uwe Doering wrote:
> With regard to the VM interaction I explained earlier, the same goes for 
> read caching.  File and directory data is kept in VM objects attached to 
> the internal vnodes (files etc.) once read in.  So large quantities of 
> disk i/o buffers aren't needed for read caching, either.
> 
> We have 'kern.nbuf="4096"' for our production systems with 2 GB RAM, 
> which results in 64 MB disk i/o cache.  These machines are used for 
> server hosting purposes and therefore run all sorts of applications at 
> the same time.  Look at the URL in my signature for more details.
> 
> I unfortunately don't know how much buffer space a dedicated database 
> server would need, but I suspect that you won't notice any difference 
> between 256 MB (default) and 64 MB (kern.nbuf="4096").
> 
> More interesting is probably to crank up 'vfs.hirunningspace' and 
> 'vfs.lorunningspace' (both 'sysctl' variables) in order to not stall 
> write operations when there are plenty of outstanding read requests 
> waiting for completion.  FreeBSD's classical bottleneck on disk i/o 
> oriented servers.  In case your raid controller has a large i/o buffer 
> of its own (16 MB or more) you may want to use values in this range:
> 
>   vfs.hirunningspace=8388608
>   vfs.lorunningspace=6291456
> 
> These are in bytes.  Also, disable agressive read-ahead in the 
> controller, if possible.  This feature is for MS Windows and would be 
> counter-productive with FreeBSD.  FreeBSD knows better than the 
> controller if and when to read ahead.
-- 
Jim C. Nasby, Database Consultant                  jim at nasby.net
Member: Triangle Fraternity, Sports Car Club of America
Give your computer some brain candy! www.distributed.net Team #1828

Windows: "Where do you want to go today?"
Linux: "Where do you want to go tomorrow?"
FreeBSD: "Are you guys coming, or what?"