How does disk caching work?

[Uwe Doering]

Jim C. Nasby wrote:
> Thanks very much for all your help. Only remaining question I have is:
> is there something we can monitor to gauge what impact (if any) changes
> to these settings have? Will changes to hi|lowrunningspace just result
> in increased KB/s or TPS write performance in gstat? How can we tell if
> we've advanced it too much?

I don't know of any simple means of monitoring the effect, short of 
doing your own benchmark tests.  If these variables are too low you will 
notice that the write throughput suffers during peak read demand. 
That's when we started to examine the kernel sources and found the reason.

To be on the safe side, don't make 'vfs.hirunningspace' larger than a 
fraction of the overall disk i/o buffer space (derived from 
'kern.nbuf'), and also not larger than the buffer space in the disk 
controller.

'vfs.lorunningspace' is used to implement a hysteresis and should be 1/2 
or 3/4 of 'vfs.hirunningspace'.

> Likewise, what can we measure regarding nbuf? If I'm understanding
> things correctly, runningspace comes out of nbuf, so obviously it needs
> to be greater than that, but what symptoms will we see if it's set too
> low?

Maybe just bad performance due to the complete flushing of the disk i/o 
buffers (remember that meta data is cached in there), maybe system 
lockup.  Can't tell.  Just make sure that these variables stay smaller 
than the disk i/o buffer cache, and you won't have to bother with the 
consequences of overdoing it.

    Uwe
-- 
Uwe Doering         |  EscapeBox - Managed On-Demand UNIX Servers
gemini at geminix.org  |  http://www.escapebox.net