zfs raidz overhead

[Karli Sjöberg]

Den 21 feb. 2017 09:46 skrev "Eugene M. Zheganin" <emz at norma.perm.ru>:
>
>
>
> Hi. 
>
> There's an interesting case described here:
> http://serverfault.com/questions/512018/strange-zfs-disk-space-usage-report-for-a-zvol
> [1] 
>
> It's a user story who encountered that under some situations zfs on
> raidz could use up to 200% of the space for a zvol. 
>
> I have also seen this. For instance: 
>
> [root at san1:~]# zfs get volsize gamestop/reference1
>  NAME PROPERTY VALUE SOURCE
>  gamestop/reference1 volsize 2,50T local
>  [root at san1:~]# zfs get all gamestop/reference1 
>  NAME PROPERTY VALUE SOURCE
>  gamestop/reference1 type volume -
>  gamestop/reference1 creation чт нояб. 24 9:09 2016 -
>  gamestop/reference1 used 4,38T -
>  gamestop/reference1 available 1,33T -
>  gamestop/reference1 referenced 4,01T -
>  gamestop/reference1 compressratio 1.00x -
>  gamestop/reference1 reservation none default
>  gamestop/reference1 volsize 2,50T local
>  gamestop/reference1 volblocksize 8K -
>  gamestop/reference1 checksum on default
>  gamestop/reference1 compression off default
>  gamestop/reference1 readonly off default
>  gamestop/reference1 copies 1 default
>  gamestop/reference1 refreservation none received
>  gamestop/reference1 primarycache all default
>  gamestop/reference1 secondarycache all default
>  gamestop/reference1 usedbysnapshots 378G -
>  gamestop/reference1 usedbydataset 4,01T -
>  gamestop/reference1 usedbychildren 0 -
>  gamestop/reference1 usedbyrefreservation 0 -
>  gamestop/reference1 logbias latency default
>  gamestop/reference1 dedup off default
>  gamestop/reference1 mlslabel -
>  gamestop/reference1 sync standard default
>  gamestop/reference1 refcompressratio 1.00x -
>  gamestop/reference1 written 4,89G -
>  gamestop/reference1 logicalused 2,72T -
>  gamestop/reference1 logicalreferenced 2,49T -
>  gamestop/reference1 volmode default default
>  gamestop/reference1 snapshot_limit none default
>  gamestop/reference1 snapshot_count none default
>  gamestop/reference1 redundant_metadata all default 
>
> [root at san1:~]# zpool status gamestop
>  pool: gamestop
>  state: ONLINE
>  scan: none requested
>  config:
>
>  NAME STATE READ WRITE CKSUM
>  gamestop ONLINE 0 0 0
>  raidz1-0 ONLINE 0 0 0
>  da6 ONLINE 0 0 0
>  da7 ONLINE 0 0 0
>  da8 ONLINE 0 0 0
>  da9 ONLINE 0 0 0
>  da11 ONLINE 0 0 0
>
>  errors: No known data errors 
>
> or, another server (overhead in this case isn't that big, but still
> considerable): 
>
> [root at san01:~]# zfs get all data/reference1
>  NAME PROPERTY VALUE SOURCE
>  data/reference1 type volume -
>  data/reference1 creation Fri Jan 6 11:23 2017 -
>  data/reference1 used 3.82T -
>  data/reference1 available 13.0T -
>  data/reference1 referenced 3.22T -
>  data/reference1 compressratio 1.00x -
>  data/reference1 reservation none default
>  data/reference1 volsize 2T local
>  data/reference1 volblocksize 8K -
>  data/reference1 checksum on default
>  data/reference1 compression off default
>  data/reference1 readonly off default
>  data/reference1 copies 1 default
>  data/reference1 refreservation none received
>  data/reference1 primarycache all default
>  data/reference1 secondarycache all default
>  data/reference1 usedbysnapshots 612G -
>  data/reference1 usedbydataset 3.22T -
>  data/reference1 usedbychildren 0 -
>  data/reference1 usedbyrefreservation 0 -
>  data/reference1 logbias latency default
>  data/reference1 dedup off default
>  data/reference1 mlslabel -
>  data/reference1 sync standard default
>  data/reference1 refcompressratio 1.00x -
>  data/reference1 written 498K -
>  data/reference1 logicalused 2.37T -
>  data/reference1 logicalreferenced 2.00T -
>  data/reference1 volmode default default
>  data/reference1 snapshot_limit none default
>  data/reference1 snapshot_count none default
>  data/reference1 redundant_metadata all default
>  [root at san01:~]# zpool status gamestop
>  pool: data
>  state: ONLINE
>  scan: none requested
>  config:
>
>  NAME STATE READ WRITE CKSUM
>  data ONLINE 0 0 0
>  raidz1-0 ONLINE 0 0 0
>  da3 ONLINE 0 0 0
>  da4 ONLINE 0 0 0
>  da5 ONLINE 0 0 0
>  da6 ONLINE 0 0 0
>  da7 ONLINE 0 0 0
>  raidz1-1 ONLINE 0 0 0
>  da8 ONLINE 0 0 0
>  da9 ONLINE 0 0 0
>  da10 ONLINE 0 0 0
>  da11 ONLINE 0 0 0
>  da12 ONLINE 0 0 0
>  raidz1-2 ONLINE 0 0 0
>  da13 ONLINE 0 0 0
>  da14 ONLINE 0 0 0
>  da15 ONLINE 0 0 0
>  da16 ONLINE 0 0 0
>  da17 ONLINE 0 0 0
>
>  errors: No known data errors 
>
> So my question is - how to avoid it ? Right now I'm experimenting with
> the volblocksize, making it around 64k. I'm also suspecting that such
> overhead may be the subsequence of the various resizing operations, like
> extening the volsize of the volume or adding new disks into the pool,
> because I have a couple of servers with raidz where the initial
> disk/volsize configuration didn't change, and the referenced/volsize
> numbers are pretty much close to each other. 
>
> Eugene. 
>
> Links:
> ------
> [1]
> http://serverfault.com/questions/512018/strange-zfs-disk-space-usage-report-for-a-zvol
> _______________________________________________
> freebsd-fs at freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-fs
> To unsubscribe, send any mail to "freebsd-fs-unsubscribe at freebsd.org"

Hello!

This problem is far from new, I investigated this back in 2013 and posted my findings here:
https://forums.freebsd.org/threads/37365/

/K