Re: Odd behaviour of two identical ZFS servers mirroring via rsync

Hi everyone,
If you have an example file that's claiming to take up more space on one
side than the other, you could use zdb to see what it's doing on disk.

e.g.
$ ls -i /workspace/mirrors/centos/CentOS-8.1.1911-x86_64-dvd1.iso
441 /workspace/mirrors/centos/CentOS-8.1.1911-x86_64-dvd1.iso
$ sudo zdb -dbdbdbdbdbdb workspace/mirrors/centos 441
Dataset workspace/mirrors/centos [ZPL], ID 1069, cr_txg 33536418, 501G,
359422 objects, rootbp DVA[0]=<3:7023204000:1000>
DVA[1]=<3:7421125000:1000> [L0 DMU objset] skein uncompressed unencrypted
LE contiguous unique double size=1000L/1000P birth=39305123L/39305123P
fill=359422
cksum=1a2c0618fec098ea:27ad9c57dd26336a:a79b9e5413f126d7:98eb32d7beb1b658

    Object  lvl   iblk   dblk  dsize  dnsize  lsize   %full  type
       441    3   128K   128K  6.83G     512  7.04G   99.99  ZFS plain file
(K=inherit) (Z=inherit=zstd-unknown)
                                               288   bonus  System
attributes
        dnode flags: USED_BYTES USERUSED_ACCOUNTED
        dnode maxblkid: 57639
        path    /CentOS-8.1.1911-x86_64-dvd1.iso
        uid     1002
        gid     1002
        atime   Sat Mar  7 19:34:54 2020
        mtime   Sat Feb 22 15:58:48 2020
        ctime   Wed Apr  8 23:11:42 2020
        crtime  Wed Apr  8 23:11:02 2020
        gen     24292265
        mode    100764
        size    7554990080
        parent  4
        links   1
        pflags  40800000004
        SA xattrs: 112 bytes, 1 entries

                user.DOSATTRIB =
0x20\000\000\003\000\003\000\000\000\021\000\000\000
\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\020\006
c\341\364\325\001\000\000\000\000\000\000\000\000
Indirect blocks:
               0 L2   DVA[0]=<2:399307d000:1000>
DVA[1]=<0:6c92a673000:1000> [L2 ZFS plain file] skein lz4 unencrypted LE
contiguous unique double size=20000L/1000P birth=33536573L/33536573P
fill=57636
cksum=4c5a8422b0199ec:8bda69b65610ddec:8ba4cc6c09a562b7:f5320ee2c5db878d
               0  L1  DVA[0]=<1:57316ff4000:b000>
DVA[1]=<2:54335f23000:b000> [L1 ZFS plain file] skein lz4 unencrypted LE
contiguous unique double size=20000L/b000P birth=33536565L/33536565P
fill=1022
cksum=23bc863675aeedbe:1ce9e654a1463229:cd17146a117928cd:cfb1524c72123546
               0   L0 DVA[0]=<2:4f75e232000:4000> [L0 ZFS plain file] skein
zstd unencrypted LE contiguous unique single size=20000L/4000P
birth=33536565L/33536565P fill=1
cksum=910008c8d6d0acfb:a5c6a7ee6f8d39de:8a5fcf7b14323a94:e5a9b7cfb00a4e98
[...]

And you can compare the block entries it prints and see why it might be
taking more space on one copy than the other...

- Rich

On Sat, Nov 12, 2022 at 4:37 AM kaycee gb <kisscoolandthegangbang@hotmail.fr>
wrote:

> Le Fri, 11 Nov 2022 17:42:44 +0000 (GMT),
> andy thomas <andy@time-domain.co.uk> a écrit :
>
> > I have two identical servers, called clustor2 and clustor-backup, each
> > with a ZFS RAIDZ-1 pool containing 9 SAS hard disks plus one spare and
> two
> > SSDs for the ZIL and ARC functions. clustor2 stores user data from a
> > HPC while clustor2-backup uses rsync to mirrors all the data from
> clustor2
> > every 24 hours.
> >
>
> Hi,
>
> For the mirroring part I would give zfs send/recv a try. I like rsync but
> I'm
> sure in this case zfs send/recv would be more efficient and faster.
>
> K.
>
>
>