ZFS file caching question

[Andrew Gallatin]

Kris Kennaway writes:
 > Andrew Gallatin wrote:
 > > If I have, say, 512MB RAM and a 1GB file which is written or read
 > > sequentially on an otherwise idle system, I'd expect the last 512MB (-
 > > epsilon) of the file's pages to be cached in RAM.  This is true for
 > > UFS, but it does not appear to be the case with ZFS (see example
 > > below).
 > > 
 > > Can somebody explain how the arc cache in ZFS relates to the normal
 > > page cache used by traditional filesystems?  Are ZFS filesystems
 > > cached exclusively in the arc cache, and not in the page cache?  Is
 > > the arc cache per-filesystem, per-pool, or global for ZFS as a whole?
 > 
 > The ZFS arc cache is completely independent from the normal buffer cache 
 > on FreeBSD.  This is inefficient in a number of ways.  I have also seen 
 > things that make me suspicious that it is not caching properly even when 
 > you tune it to be "large enough" (if possible given memory constraints), 
 > but I haven't confirmed this.

In some ways this is kind of cool.  I'm want to use FreeBSD+ZFS for a
new desktop which will also host a media server.  If I put the media
on ZFS, and my home directory on UFS, then the gigantic HD media files
recorded overnight won't push my desktop applications out of RAM
overnight.  That's a feature :)

 > > Hmm.. Could this be the cause of the problems with ZFS and mmap'ed files?
 > 
 > What problems do you mean?  There were coherency problems but I think 
 > they were fixed.

The last I saw about this on -current was:
http://lists.freebsd.org/pipermail/freebsd-current/2008-February/083396.html

This seemed to terminate with people disabling mmap in the
applications, not with a fix to ZFS.  Maybe I missed a commit..

Thanks,

Drew