copying files between zfs servers results in different data in these files
Thorsten Schlich
thorsten.schlich at wetteronline.de
Mon Jul 7 08:58:32 UTC 2014
Hello,
i have a somehow crazy behaviour in copying files from one zfs server to
another.
The files seem to be the same (same size, same timestamp, etc.) but
after copying the binary data differs at some data points between the
original and the copy.
ZFS is on both servers the standard filesystem.
server 1 is a freebsd 8.3 vm (update is in progress but for produtional
reasons it needs some time) with a zpool named tank including one
virtual disk.
server 2 is a freebsd 9.2 hardware machine with 6x3 TB in raidz (1-0)
pool "tank"
The copy process is:
- find ./ -type f -atime +2 >/tmp/file.txt
- rsync the files named in /tmp/file.txt to server 2
For one day there are 15682 files copied and between 9000 and 10000
copied files differ from their original.
The difference is small. For 21.35 in the original there is a 21.85 in
the copy. But without any pattern (checked).As this are meteorological
data such a minor change is crucial to every calculation which comes later.
Copying between two VMs is no problem, all files are correct.
Only the machine with the raidz has this problem. Additionally the raidz
machine boots with a usb drive.
Both server are in the same room wired in the same network and can
access each other directly.
The version of rsync is all the same and zfs is the same version too.
Do you have any hints or suggestions where i could investigate further?
Thanks in advance for help. Below you can find the zpool and zfs
configuration.
Regards,
Thorsten
***
Configuration Server 1:
zfs get all tank
NAME PROPERTY VALUE SOURCE
tank type filesystem -
tank creation Mo Jan 13 15:40 2014 -
tank used 437G -
tank available 193G -
tank referenced 437G -
tank compressratio 1.78x -
tank mounted yes -
tank quota none default
tank reservation none default
tank recordsize 128K default
tank mountpoint /space local
tank sharenfs off default
tank checksum on default
tank compression on local
tank atime on default
tank devices on default
tank exec on default
tank setuid on default
tank readonly off default
tank jailed off default
tank snapdir hidden default
tank aclmode passthrough local
tank aclinherit passthrough local
tank canmount on default
tank xattr off temporary
tank copies 1 default
tank version 5 -
tank utf8only off -
tank normalization none -
tank casesensitivity sensitive -
tank vscan off default
tank nbmand off default
tank sharesmb off default
tank refquota none default
tank refreservation none default
tank primarycache all default
tank secondarycache all default
tank usedbysnapshots 0 -
tank usedbydataset 437G -
tank usedbychildren 322M -
tank usedbyrefreservation 0 -
tank logbias latency default
tank dedup off default
tank mlslabel -
tank sync standard default
tank refcompressratio 1.78x -
tank written 437G -
zpool get all tank
NAME PROPERTY VALUE SOURCE
tank size 640G -
tank capacity 68% -
tank altroot - default
tank health ONLINE -
tank guid 7522257494086463050 default
tank version 28 default
tank bootfs - default
tank delegation on default
tank autoreplace off default
tank cachefile - default
tank failmode wait default
tank listsnapshots off default
tank autoexpand on local
tank dedupditto 0 default
tank dedupratio 1.00x -
tank free 203G -
tank allocated 437G -
tank readonly off -
tank comment - default
***
Configuration Server 2:
zfs get all tank
NAME PROPERTY VALUE SOURCE
tank type filesystem -
tank creation Fr Feb 14 8:13 2014 -
tank used 12,7T -
tank available 5,10T -
tank referenced 683M -
tank compressratio 1.74x -
tank mounted yes -
tank quota none default
tank reservation none default
tank recordsize 128K default
tank mountpoint /space local
tank sharenfs off default
tank checksum on default
tank compression gzip-9 local
tank atime on default
tank devices on default
tank exec on default
tank setuid on default
tank readonly off default
tank jailed off default
tank snapdir hidden default
tank aclmode discard default
tank aclinherit restricted default
tank canmount on default
tank xattr off temporary
tank copies 1 default
tank version 5 -
tank utf8only off -
tank normalization none -
tank casesensitivity sensitive -
tank vscan off default
tank nbmand off default
tank sharesmb off default
tank refquota none default
tank refreservation none default
tank primarycache all default
tank secondarycache all default
tank usedbysnapshots 0 -
tank usedbydataset 683M -
tank usedbychildren 12,7T -
tank usedbyrefreservation 0 -
tank logbias latency default
tank dedup off default
tank mlslabel -
tank sync standard default
tank refcompressratio 1.94x -
tank written 683M -
tank logicalused 21,9T -
tank logicalreferenced 1,28G -
zpool get all tank
NAME PROPERTY VALUE SOURCE
tank size 21,8T -
tank capacity 70% -
tank altroot - default
tank health ONLINE -
tank guid 4365850585010436054 default
tank version - default
tank bootfs - default
tank delegation on default
tank autoreplace off default
tank cachefile - default
tank failmode wait default
tank listsnapshots off default
tank autoexpand on local
tank dedupditto 0 default
tank dedupratio 1.00x -
tank free 6,49T -
tank allocated 15,3T -
tank readonly off -
tank comment - default
tank expandsize 0 -
tank freeing 0 default
tank feature at async_destroy enabled local
tank feature at empty_bpobj active local
tank feature at lz4_compress enabled local
More information about the freebsd-fs
mailing list