ZFS 4k aligned space overhead

[Adam Nowacki]

Free space calculation is done with the assumption of 128k block size. 
Each block is completely independent so sector aligned and no parity 
shared between blocks. This creates overhead unless the number of disks 
minus raidz level is a power of two. Above that is allocation overhead 
where each block (together with parity) is padded to occupy the multiple 
of raidz level plus 1 (sectors). Zero overhead from both happens at 
raidz1 with 2, 3, 5, 9 and 17 disks and raidz2 with 3, 6 or 18 disks.

On 2013-05-29 15:18, Hans Henrik Happe wrote:
> Hi,
>
> I've a system with 3TB WD NAS disks for ZFS. I created a 4k aligned 10
> disk RAIDZ2. I noticed the overhead was ~1.4TB for the file system
> (3*8*10^12/2^40 - <free space>). Then I tried with different number of
> disks:
>
> 6: 0.2602885812520981
> 7: 1.1858475767076015
> 8: 1.149622268974781
> 9: 0.7288754601031542
> 10: 1.3953630803152919
> 11: 2.061850775964558
> 12: 2.915792594663799
> 13: 1.5491229379549623
> 14: 2.056995471008122
> 15: 2.5648680040612817
> 16: 3.0727405650541186
> 17: 3.5806130981072783
> 18: 0.7912140190601349
>
> the other good configs (6 and 18) is okay, but it seem strange that 10
> has higher spaceoverhead than 18.

High overhead with 10 disks is because of allocation overhead.
128k / 4k = 32 sectors,
32 sectors / 8 data disks = 4 sectors per disk,
4 sectors per disk * (8 data disks + 2 parity disks) = 40 sectors.
40 is not a multiple of 3 so 2 sector padding is added. (5% overhead)

>
> I then tried with RAIDZ:
>
> 5: 0.19996666628867388
> 9: 0.39560710452497005
> 17: 0.8849408514797688
>
> This seems correct. Then RAIDZ2 again but with ashift=9:
>
> 6:  0.2602881230413914
> 10: 0.4523236369714141
> 18: 0.7912133224308491
>
> This also seems correct. The 6 and 18 results are basically the same for
> ashift 9 and 12.
>
> Is there an explanation to this?
>
> I'm running FreeBSD 9.1-RELEASE-p3.
>
> Cheers,
> Hans Henrik Happe
> _______________________________________________
> freebsd-fs at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-fs
> To unsubscribe, send any mail to "freebsd-fs-unsubscribe at freebsd.org"