ZFS - poor performance with "large" directories

[Adam Vande More]

On Tue, Nov 24, 2015 at 8:00 AM, Albert Cervin <albert at acervin.com> wrote:

> Hi all,
>
> Please feel free to direct me to a list that is more suitable.
>
> We are trying to set up a fileserver solution for a web application that we
> are building. This fileserver is running FreeBSD 10.2 and ZFS. Files are
> written over CIFS with Samba running on the fileserver host.
>
> However, we are seeing en exponential decrease in performance to write to
> the file server when the number of files in the directory grows (when it
> goes up to ~6000 files it becomes unusable and the write time has gone from
> a fraction of a second to ten seconds).
>
> We ran the same setup on a Linux machine with an ext4 file system which did
> NOT suffer from this performance degradation.
>

I should hope not.  ext4 vs zfs comparison isn't fair for either.


>
> Are these "holes" in write speed normal. Since this is the exact symptom we
> are getting when the network writes start to be slow.
>

Totally normal.  You'll want to reference:

https://wiki.freebsd.org/ZFSTuningGuide

In particular for that issue see:

vfs.zfs.txg.timeout
and tuning related to NFS.

Performance is also heavily dependent on pool structure and io
characteristics.  For example, a pool of 3 2 disk mirrors is in general
going to be much faster than 1 6 disk raidz2.


-- 
Adam