ZFS vs. df -h completely different size of filesystem
Miroslav Lachman
000.fbsd at quip.cz
Sun Oct 11 04:39:13 UTC 2009
Hi,
I am using ZFS on this machine for one year without any problems. The
main purpose is to run jails on filesystems with ZFS quotas.
The biggest jail has 360G (322GB used) running Lighttpd. I must change
IP for this jail, so I stop the jail, asign new IP alias on interface,
change IP in config and start the jail again. But now the jail reports
only 24G quota and by another accident I deleted stored files by rsync.
The strange thing is this:
root at cage ~/# df -h /vol0/jail/kurt2
Filesystem Size Used Avail Capacity Mounted on
tank/vol0/jail/kurt2 24G 19G 4.9G 80% /vol0/jail/kurt2
root at cage ~/# zfs list | grep tank/vol0/jail/kurt2
tank/vol0/jail/kurt2 355G 4.89G 19.2G
/vol0/jail/kurt2
tank/vol0/jail/kurt2 at manual-2009-08-18_11-04 7.29G - 299G -
tank/vol0/jail/kurt2 at manual-2009-09-04_10-24 2.24G - 308G -
tank/vol0/jail/kurt2 at manual-2009-09-28_11-41 21.4G - 317G -
root at cage ~/# zfs get -r quota tank/vol0/jail/kurt2
NAME PROPERTY VALUE SOURCE
tank/vol0/jail/kurt2 quota 360G local
As you can see, df reports 24G size and zfs list shows 355G.
What is wrong? I tired to write some date on to this filesystem and it
stoped after 24G were filled.
Here are more details:
root at cage ~/# zfs get all tank/vol0/jail/kurt2
NAME PROPERTY VALUE SOURCE
tank/vol0/jail/kurt2 type filesystem -
tank/vol0/jail/kurt2 creation Sat Feb 28 0:52 2009 -
tank/vol0/jail/kurt2 used 355G -
tank/vol0/jail/kurt2 available 4.89G -
tank/vol0/jail/kurt2 referenced 19.2G -
tank/vol0/jail/kurt2 compressratio 1.02x -
tank/vol0/jail/kurt2 mounted yes -
tank/vol0/jail/kurt2 quota 360G local
tank/vol0/jail/kurt2 reservation none default
tank/vol0/jail/kurt2 recordsize 128K default
tank/vol0/jail/kurt2 mountpoint /vol0/jail/kurt2 inherited
from tank/vol0
tank/vol0/jail/kurt2 sharenfs off default
tank/vol0/jail/kurt2 checksum on default
tank/vol0/jail/kurt2 compression gzip inherited
from tank/vol0
tank/vol0/jail/kurt2 atime off inherited
from tank
tank/vol0/jail/kurt2 devices on default
tank/vol0/jail/kurt2 exec on inherited
from tank/vol0/jail
tank/vol0/jail/kurt2 setuid on inherited
from tank/vol0/jail
tank/vol0/jail/kurt2 readonly off default
tank/vol0/jail/kurt2 jailed off default
tank/vol0/jail/kurt2 snapdir hidden default
tank/vol0/jail/kurt2 aclmode groupmask default
tank/vol0/jail/kurt2 aclinherit secure default
tank/vol0/jail/kurt2 canmount on default
tank/vol0/jail/kurt2 shareiscsi off default
tank/vol0/jail/kurt2 xattr off temporary
tank/vol0/jail/kurt2 copies 1 default
root at cage ~/# df -ht zfs
Filesystem Size Used Avail Capacity Mounted on
tank 76G 16G 59G 22% /tank
tank/vol0 59G 128K 59G 0% /vol0
tank/vol0/jail 60G 605M 59G 1% /vol0/jail
tank/vol0/jail/china 20G 786M 19G 4%
/vol0/jail/china
tank/vol0/jail/costa 40G 527M 39G 1%
/vol0/jail/costa
tank/vol0/jail/kurt2 24G 19G 4.9G 80%
/vol0/jail/kurt2
tank/vol0/jail/tester1 20G 874M 19G 4%
/vol0/jail/tester1
tank/vol0/jail/tvujweb 25G 494M 24G 2%
/vol0/jail/tvujweb
tank/vol0/jail/yamaha 19G 1.2G 18G 6%
/vol0/jail/yamaha
tank/vol0/jail/kurt2_clone 376G 317G 59G 84%
/vol0/jail/kurt2_clone
root at cage ~/# zfs list -t filesystem
NAME USED AVAIL REFER MOUNTPOINT
tank 378G 59.2G 16.4G /tank
tank/vol0 361G 59.2G 19K /vol0
tank/vol0/jail 361G 59.2G 605M /vol0/jail
tank/vol0/jail/china 996M 19.0G 786M /vol0/jail/china
tank/vol0/jail/costa 938M 39.1G 527M /vol0/jail/costa
tank/vol0/jail/kurt2 355G 4.89G 19.2G /vol0/jail/kurt2
tank/vol0/jail/kurt2_clone 0 59.2G 317G /vol0/jail/kurt2_clone
tank/vol0/jail/tester1 1.23G 18.8G 874M /vol0/jail/tester1
tank/vol0/jail/tvujweb 691M 24.3G 494M /vol0/jail/tvujweb
tank/vol0/jail/yamaha 1.72G 18.3G 1.18G /vol0/jail/yamaha
kurt2_clone was created as clone of snapshot
kurt2 at manual-2009-09-28_11-41 after this crazyness
Did somebody seen this error behavior?
I can get all the data from master server by rsync, but 320GB of data,
it will take many hours. Is there any way to move the data from
kurt2_clone to kurt2? (after I find a reason, why kurt2 is so small now)
I cannot use promote:
root at cage ~/# zfs promote tank/vol0/jail/kurt2_clone
cannot promote 'tank/vol0/jail/kurt2_clone': out of space
system: FreeBSD 7.2-RELEASE #1: Tue May 5 16:17:50 CEST 2009
root at cage ~/# zdb
tank
version=6
name='tank'
state=0
txg=4
pool_guid=15882109565948440736
hostid=2083894993
hostname='cage.xxxxx.yyy'
vdev_tree
type='root'
id=0
guid=15882109565948440736
children[0]
type='mirror'
id=0
guid=15485397551239445092
metaslab_array=14
metaslab_shift=32
ashift=9
asize=478648991744
children[0]
type='disk'
id=0
guid=7324230677133648509
path='/dev/ad4s2'
devid='ad:GEB531RE0B213Fs1'
whole_disk=0
children[1]
type='disk'
id=1
guid=16876684646990874563
path='/dev/ad6s2'
devid='ad:GEB531RE0BJ72Fs1'
whole_disk=0
Miroslav Lachman
More information about the freebsd-fs
mailing list