SU+J: negative used diskspace (for a while)

[Hans Ottevanger]

On Tue, Jun 21, 2011 at 06:46:45PM +1000, Peter Jeremy wrote:
> On 2011-Jun-17 11:42:26 -0700, Kirk McKusick <mckusick at mckusick.com> wrote:
> >> Date: Fri, 17 Jun 2011 17:34:15 +0200
> >> From: Hans Ottevanger <hans at beastielabs.net>
> >> So it takes more than a minute before the disk space is back to "normal"
> >> values.
> 
> >We used to account for deleted blocks at the instant that they were
> >removed. This accounting was rather complex, so as part of doing
> >SU+J, Jeff simplified it. Under the simplification, the removal is
> >not accounted for until part way through the removal process. The
> >result is that you now get these false negative block counts until
> >the blocks have been partially reclaimed. If this behavior causes
> >enough trouble, Jeff might be convinced that the more accurate block
> >accounting is necessary.
> 
> Negative values may also impact NFS clients - though just limiting the
> reported used space to 0 should avoid them getting too upset.
> 

It appears that negative values occur only when the filesystem is almost
empty (like in my tests). Otherwise the values are just way too low for
a while.

> That said, whilst I haven't seen negative used values, ZFS and
> Solaris UFS also take an extended period before 'df' reports
> correct values (several minutes for Solaris UFS).  In the case
> of ZFS, even 'du' can report incorrect information for a while.
> 

If I remember well, in "the old days", when Soft Updates was fairly new,
it also had such issues in the first release, but it was repaired soon
afterwards.

Of course it is a real advantage to see the results immediately when you
are cleaning out filesystems, especially when you are in a hurry.

I just verified that the current issue was quite recently introduced with
commit r222958. Before that 'df' reported changes promptly and correctly,
even with Journaling enabled.

As an answer to my original message to -current@ Peter Holm has assured me
(http://lists.freebsd.org/pipermail/freebsd-current/2011-June/025318.html)
that this is a known issue that is on Jeff's TODO list.

All I can say to the UFS+J guys is: keep up the good work. Journaling is a
very cool addition to UFS, amongst others a real time saver at boot time
after system crashes. Just a few years ago I saw people writing that adding
journaling to UFS was not really possible and it would probably never happen.
Well, here it is, though I understand that it was a bit more difficult
than foreseen and still needs a lot of work 8-). Thanks, guys.

Kind regards,

Hans Ottevanger