ZFS and large directories - caveat report

[Willem Jan Withagen]

On 21-7-2011 19:18, Ivan Voras wrote:
> On 21 July 2011 18:38, Luiz Otavio O Souza <lists.br at gmail.com> wrote:
> 
>> The general usage on this server is fine, but the periodic (daily) scripts take almost a day to complete and the server is slow as hell while the daily scripts are running.
> 
> Yes, this is how my problem was first diagnosed.
> 
>> So, yes, i can confirm that running 'find' on a ZFS FS with a lot of files is very, very slow (and looks like it isn't related to how the files are distributed on the FS).
> 
> Only it's not just "find" - it's any directory operations - including
> file creation and removal. I cannot say that is not related to how
> files are distributed on the file system, except the unusually long
> operations on the parent of the shard directories in my case.

A little late in the thread:

Running on 8.2-stable, ZFS version 15
Quad core, 8Gb memory, /home is on a 6-disk(sata) raidz2 fs.
The dicertory is a 3 week revolving log of images taken from a security
com. So if anything it directory-file should be horribly thrashed
It is around 170.000 files in one directory.

[/home/sonycam] wjw at zfs.digiware.nl> ls periodical | wc
  177364  177364 3369916
0.421u 6.999s 2:00.15 6.1%      37+1522k 0+0io 0pf+0w
[/home/sonycam] wjw at zfs.digiware.nl> ls -asl periodical | wc
  177401 1774002 13659785
1.747u 11.087s 1:42.98 12.4%    36+1562k 0+0io 0pf+0w

Repeated finds after this complete with 10 secs.

On average I seen about 100 IOPS/disk and readin is at 5Mbyte/disk.

But I do not feel the system is really loaded while doing the ls.
I can easily login again, and do other work.

--WjW