ZFS extra space overhead for ashift=12 vs ashift=9 raidz2 pool?

[Taylor]

Alex,

I think you are correct. It occurred to me some time after reading your original email that the sector size problem
could also be applied to the metadata for the filesystem as well as the data. As I previously stated, the overhead
of the filesystem goes from 2.59% to 8.06% when increasing sector size from 512B to 4KiB , which is an increase of 3.11x,
well in line with your 8x observation. Likewise this thread also seems to confirm that lots of the metadata takes up
< 512B and there is no real attempts to optimize this for 4K sector size:
http://mail.opensolaris.org/pipermail/zfs-discuss/2011-October/049959.html

I ended up using 512B sector size for the array since I valued the extra space more than the extra bandwidth. :) 

Thanks again for your response,

-Taylor


On Apr 2, 2012, at 4:37 AM, Alexander Leidinger wrote:

> Quoting Taylor <j.freebsd-zfs at enone.net> (from Sat, 24 Mar 2012 11:41:20 -0700):
> 
>> Alex,
>> 
>> Thank you for your response. I'm not particularly concerned about the overhead of file fragmentation,
>> as most of the space will be take by fairly large files (10's of GiB).
>> 
>> My original question concerned the amount of space reported available by zfs for a
>> freshly-created *empty* raidz2 filesystem.
>> 
>> To re-iterate, I find 2.79TiB  more space available with ashift=9 (49.62 TiB) vs ashift=12 (46.83TiB)
>> for a new 3.64TiB 16-disk raidz2 pool.
> 
> I do not know for the actual amount, but at least some overhead is not surprising to me.
> 
> You have some meta data in ZFS (file permissions, ACLs, checksums, ...). This meta data should be more often much less than 4k in size, but you need to allocate at least one block for this meta data. If we assume (worst case) that most of the time the meta data would fit into 512 byte but you always use a 4k sector, it should be clear that you use 8 times more space on the disk for each meta data unit, than necessary.
> 
> Bye,
> Alexander.
> 
> -- 
> Let me put it this way: today is going to be a learning experience.
> 
> http://www.Leidinger.net    Alexander @ Leidinger.net: PGP ID = B0063FE7
> http://www.FreeBSD.org       netchild @ FreeBSD.org  : PGP ID = 72077137
>