bootloader block cache improvement

[John Baldwin]

On Thursday, September 22, 2011 2:02:22 pm Artem Belevich wrote:
> Hi,
> 
> I've had ZFS-only box that boots off 8-drive raidz2 array. I've noticed
> that on this machine it takes noticeably longer to load kernel and
> modules than on a similar box that boots off 1-drive ZFS filesystem.
> 
> It turns out that block cache in loader only caches data from one disk
> only and invalidates the cache as soon as we read from another
> drive. With ZFS reading from multiple drives when filesystem is on a
> raidz pool the cache was effectively useless in that scenario. I've got
> literally 0 hits reported by bcachestat command.

One thing to keep in mind is that the cache was designed to optimize the 
experience with floppy disks.  There is one implication from this which is 
non-obvious.  Floppy drives do not have a reliable signal for "the disk has 
changed", so the loader flushes the cache on certain operations (file
close, or at least it used to).

I think the logic to flush when the unit changed is what provided that 
feature, but it's not as obvious now.  It would be nice to still flush the 
cache for floppies, but I'm not sure how feasible that would be to maintain.

At the very least you could perhaps add a bcache_flush_unit() and call that
from bd_closedisk() in libi386/biosdisk.c if OD_FLAGS_FLOPPY is set.

Aside from that, I think the patch is good.

-- 
John Baldwin