Millions of small files: best filesystem / best options

[Alessio Focardi]

> FreeBSD handles this wonderfully thanks to all the people that have
> put in time and effort over the years.

That's a great news and I recognize the effort of the community!
 
> I wasn't aware that FreeBSD was unique in this respect, but yes,
> FreeBSD has a block size and a fragment size. While formatting a UFS
> filesystem you can specify these sizes with the "-b SIZE" and "-f
> SIZE" arguments to newfs(8), for example:
> 
> 	newfs -b 16384 -f 2048 /dev/da0s1a

-b block-size
	     The block size of the file system, in bytes.  It must be a power
	     of 2.  The default size is 16384 bytes, and the smallest allow-
	     able size is 4096 bytes.

-f frag-size
	     The fragment size of the file system in bytes.  It must be a
	     power of two ranging in value between blocksize/8 and blocksize.
	     The default is 2048 bytes.

So in my case I would have to use -b 4096 -f 512

It's an improvement, but still is not ideal: still a big waste with 200 bytes files!

ZFS with compression, maybe?


> Choose your hardware wisely. After you have chosen your hardware
> wisely, set it up even more wisely.

That's a good advice! I'm still working on the theory of the system, trying to find a solution for the "slack" problem, then it will come the time to look at a storage platform and surely we will choose something we can trust!

Tnx for you reply, informative and well written!