ZFS and large directories - caveat report

[Martin Matuska]

On 21. 7. 2011 21:40, Ivan Voras  wrote:
> Thank you very much - now if only you took as much effort to explain
> the possible connection between your quote and my post as it took you
> to find the quote :)
>
> As others explained, ZFS definitely does not use fixed block sizes
I agree to that.

I tried some more digging and stomped on this opensolaris mailing list
thread:

It starts here:
http://www.mail-archive.com/zfs-discuss@opensolaris.org/msg35150.html
With an interesting user report here (nice summary):
http://www.mail-archive.com/zfs-discuss@opensolaris.org/msg35189.html

Most times they blame the way the client utilities work with directories
(sorting etc.).

Now to some more relevant options:
It is also possible to do metadata-only caching for a dataset:
zfs set primarycache=metadata
L2 can be modified as well: zfs set secondarycache=metadata
If I find some time I can run some simulations on this to see how it
performs compared to primarycache=all.

The vdev read-ahead cache might also have a negative impact here (lots
of wasted IOPS), mostly if the blocks are spread around the vdev.
We have followed what Illumos did and vdev cache is now disabled by default.

I have updated the zfs-stats tool (ports: sysutils/zfs-stats) with
latest Jason J. Hellenthal's arc_summary.pl, it gives a good overview of
ZFS sysctl's:
https://github.com/mmatuska/zfs-stats

-- 
Martin Matuska
FreeBSD committer
http://blog.vx.sk