ZFS directory with a large number of files

[Patrick M. Hausen]

Hi, all,

Am 05.08.2011 um 17:12 schrieb Christian Weisgerber:
> Daniel Kalchev <daniel at digsys.bg> wrote:
> 
>> On 02.08.11 12:46, Daniel O'Connor wrote:
>>> I am pretty sure UFS does not have this problem. i.e. once you 
>>> delete/move the files out of the directory its performance would be 
>>> good again. 
>> 
>> UFS would be the classic example of poor performance if you do this.
> 
> "Classic" indeed.  UFS dirhash has pretty much taken care of this
> a decade ago.

While dirhash is quite an improvement, it is definitely no silver bullet.

When I asked Kirk McKusick at last year's EuroBSDCon if having
a six-figure number of files in a single directory was a clever idea
(I just had a customer who ran into that situation), he just smiled
and shook his head.

The directory in question was the typo3temp/pics directory that
TYPO3 uses to scale images that are part of the website, so they
are handed to the browser in exactly the size they are supposed
to be rendered. The performance impact was quite heavy, because
at some point requests started to pile up, PHP scripts did not finish
in time, fcgi slots stayed used ... most of you will know that scenario.
At some threshold a machine goes from "loaded, maybe a bit slow,
but generally responsive" to "no f*ing way".

Best regards,
Patrick