snapshots and disk usage

[Dan Nelson]

In the last episode (Sep 07), Stefan `Sec` Zehl said:
> Hi,
> 
> I am using ufs snapshots on RELENG_7 for some time now, and am generally
> happy with it. I have noticed a strange behaviour when removing large
> amount of files, and wanted to ask if this is expected.
> 
> Before starting, we check the free space on /usr:
> 
> | ice:/usr>df -h .
> | Filesystem         Size    Used   Avail Capacity  Mounted on
> | /dev/ad4s2.elid    9.7G    7.6G    1.3G    64%    /usr
> 
> Then delete /usr/obj and run df again:
> 
> | ice:/usr>sudo rm -rf obj 2>/dev/null
> | ice:/usr>df -h .
> | Filesystem         Size    Used   Avail Capacity  Mounted on
> | /dev/ad4s2.elid    9.7G    5.7G    3.2G    64%    /usr
> 
> This is unexpected. With snapshots, removing something should not
> release space.
> 
> Sure enough, in the course of the next minute, the fake free space
> vanishes....
> 
> | ice:/usr>df -h .
> | Filesystem         Size    Used   Avail Capacity  Mounted on
> | /dev/ad4s2.elid    9.7G    5.9G    3.0G    66%    /usr
> | ice:/usr>df -h .
> | Filesystem         Size    Used   Avail Capacity  Mounted on
> | /dev/ad4s2.elid    9.7G    6.6G    2.3G    74%    /usr
> | ice:/usr>df -h .
> | Filesystem         Size    Used   Avail Capacity  Mounted on
> | /dev/ad4s2.elid    9.7G    8.6G    269M    97%    /usr
> 
> and all the free space is allocated in the snapshot:
> 
> | ice:~>sudo snapshot list
> | Filesystem          User   User%     Snap   Snap%  Snapshot
> | /usr                 8GB   89.3%      2GB   21.5%  daily.1
> | /usr                 8GB   89.3%    344MB    3.5%  daily.0
> | /usr                 8GB   89.3%    344MB    3.5%  weekly.0
> | /usr                 8GB   89.3%    344MB    3.5%  hourly.1
> | /usr                 8GB   89.3%      7MB    0.1%  hourly.0
> 
> My understanding so far was that df may underreport free space, but i
> find overreporting it a bit troublesome. -- What would happen if I tried
> to use that space before it was allocated to the snapshot?

I think you're running into the softupdates delay.  When you delete a
file on a SU-enabled filessytem, the space isn't actually freed until
sync.  But applications expect that statfs() info is updated
immediately, so the kernel pretends that the space is available.  That
doesn't really work with a snapshot, since if you delete a file that
existed in the snapshot, no space will free up.  So you see a jump in
freespace as the kernel fakes the f_bfree statfs amount, then it slowly
drops to the correct value as the deletions actually sync to disk.

-- 
	Dan Nelson
	dnelson at allantgroup.com