disk space not freed when deleting files in null_mounted filesystem

[Konstantin Belousov]

On Thu, Jun 09, 2016 at 02:11:26PM +0200, Peter Holm wrote:
> On Thu, Jun 09, 2016 at 01:39:45PM +0200, richard bader wrote:
> > Am 09.06.2016 um 13:27 schrieb Peter Holm:
> > > On Thu, Jun 09, 2016 at 12:13:08PM +0200, richard bader wrote:
> > >> Hello,
> > >>
> > >> i've a filesystem mounted with mount_nullfs.
> > >> When i delete a large file in this filesystem, the space on the disk is
> > >> not freed.
> > >> The space becomes freed when i unmount the filesystem.
> > >>
> > >>
> > >> to reproduce this:
> > >>
> > >> root at main2:/tmp # mkdir lnk src
> > >> root at main2:/tmp # mount_nullfs src lnk
> > >> root at main2:/tmp # mount | grep lnk
> > >> /tmp/src on /tmp/lnk (nullfs, local)
> > >> root at main2:/tmp # df -h .
> > >> Filesystem     Size    Used   Avail Capacity  Mounted on
> > >> /dev/ada0p2     58G     40G     13G    75%    /
> > >> root at main2:/tmp # dd if=/dev/zero of=lnk/foo bs=1G count=3
> > >> 3+0 records in
> > >> 3+0 records out
> > >> 3221225472 bytes transferred in 7.856793 secs (409992411 bytes/sec)
> > >> root at main2:/tmp # df -h .
> > >> Filesystem     Size    Used   Avail Capacity  Mounted on
> > >> /dev/ada0p2     58G     43G     10G    80%    /
> > >> root at main2:/tmp # rm lnk/foo
> > >> root at main2:/tmp # df -h .
> > >> Filesystem     Size    Used   Avail Capacity  Mounted on
> > >> /dev/ada0p2     58G     43G     10G    80%    /
> > >> root at main2:/tmp # umount /tmp/lnk
> > >> root at main2:/tmp # df -h .
> > >> Filesystem     Size    Used   Avail Capacity  Mounted on
> > >> /dev/ada0p2     58G     40G     13G    75%    /
> > >>
> > >>
> > >> my system:
> > >>
> > >> root at main2:/tmp # uname -a
> > >> FreeBSD main2.bader.loc 10.2-RELEASE-p18 FreeBSD 10.2-RELEASE-p18 #8
> > >> r301112: Wed Jun  1 11:08:24 CEST 2016
> > >> root at main2.bader.loc:/usr/obj/usr/src/sys/MYKERNEL  amd64
> > >>
> > >> are the any options for mount_nullfs ?
> > >>
> > >> best regards
> > >>
> > >> richard
> > >>
> > > Hello Richard,
> > >
> > > I wonder which file system you are using?
> > > Here's what happens with UFS and current from today:
> > >
> > > $ cat -n richard.sh
> > >       1  #!/bin/sh
> > >       2
> > >       3  uname -a
> > >       4  mount | grep `df -h /tmp | tail -1 | awk '{print $1}'`
> > >       5
> > >       6  cd /tmp
> > >       7  rm -rf lnk src
> > >       8  mkdir lnk src
> > >       9  mount_nullfs src lnk
> > >      10  mount | grep lnk
> > >      11  df -h .
> > >      12  dd if=/dev/zero of=lnk/foo bs=1G count=3
> > >      13  df -h .
> > >      14  rm lnk/foo
> > >      15  df -h .
> > >      16  umount /tmp/lnk
> > >      17  df -h .
> > > $ ./richard.sh
> > > FreeBSD t2.osted.lan 11.0-ALPHA2 FreeBSD 11.0-ALPHA2 #0 r301722: Thu Jun  9 12:09:39 CEST 2016     pho at t2.osted.lan:/usr/src/sys/amd64/compile/PHO  amd64
> > > /dev/da0p3 on /tmp (ufs, NFS exported, local, soft-updates)
> > > /tmp/src on /tmp/lnk (nullfs, local)
> > > Filesystem    Size    Used   Avail Capacity  Mounted on
> > > /dev/da0p3     97G    1,6G     87G     2%    /tmp
> > > 3+0 records in
> > > 3+0 records out
> > > 3221225472 bytes transferred in 47.658947 secs (67589102 bytes/sec)
> > > Filesystem    Size    Used   Avail Capacity  Mounted on
> > > /dev/da0p3     97G    4,6G     84G     5%    /tmp
> > > Filesystem    Size    Used   Avail Capacity  Mounted on
> > > /dev/da0p3     97G    1,6G     87G     2%    /tmp
> > > Filesystem    Size    Used   Avail Capacity  Mounted on
> > > /dev/da0p3     97G    1,6G     87G     2%    /tmp
> > > $
> > >
> > > - Peter
> > i'm also using ufs.
> > 
> > root at main2:/tmp # ./richard.sh
> > FreeBSD main2.bader.loc 10.2-RELEASE-p18 FreeBSD 10.2-RELEASE-p18 #8 
> > r301112: Wed Jun  1 11:08:24 CEST 2016 
> > root at main2.bader.loc:/usr/obj/usr/src/sys/MYKERNEL  amd64
> > /dev/ada0p2 on / (ufs, local, journaled soft-updates)
> > /tmp/src on /tmp/lnk (nullfs, local)
> > Filesystem     Size    Used   Avail Capacity  Mounted on
> > /dev/ada0p2     58G     40G     13G    75%    /
> > 3+0 records in
> > 3+0 records out
> > 3221225472 bytes transferred in 8.679717 secs (371121023 bytes/sec)
> > Filesystem     Size    Used   Avail Capacity  Mounted on
> > /dev/ada0p2     58G     43G     10G    80%    /
> > Filesystem     Size    Used   Avail Capacity  Mounted on
> > /dev/ada0p2     58G     43G     10G    80%    /
> > Filesystem     Size    Used   Avail Capacity  Mounted on
> > /dev/ada0p2     58G     40G     13G    75%    /
> > 
> 
> Hmm. I tried again on SU+J but I still do not see any problems on
> HEAD. I'll build r301112 and see what happens.

There were several bugs fixed since 10.2 in nullfs.  Use -o nocache
to disable nullfs vnode caching, which most likely fix it for 10.2.