nullfs with double mount is broken in FreeBSD 10 Stable

[Konstantin Belousov]

On Tue, Sep 19, 2017 at 11:33:18AM +0200, Andreas Longwitz wrote:
> The described problem does not exist in V8, but in V10 Stable r317936.
> 
> I use a device /dev/md2 defined in /etc/rc.conf:
> 
> mdconfig_md2="-t swap -s 384m"
> mdconfig_md2_newfs="-n"
> mdconfig_md2_owner="root:wheel"
> mdconfig_md2_perms="750"
> 
> In /etc/fstab I have:
> 
>  /dev/md2   /tmp1       ufs     rw,async,noatime,noauto 0  0
>  /tmp1      /home/tmp1  nullfs  rw,noauto               0  0
>  /tmp1      /var/tmp1   nullfs  rw,noauto               0  0
> 
> Now I run:
> 
> mount /home/tmp1
> mount /var/tmp1
> while true; do
>    rm -f /home/tmp1/*
>    cpdup /boot/kernel /home/tmp1
>    cp /var/tmp1/kernel /dev/null
>    df -h /tmp1
> done
> 
> The output is
> 
> Filesystem    Size    Used   Avail Capacity  Mounted on
> /dev/md2      372M     88M    254M    26%    /tmp1
> Filesystem    Size    Used   Avail Capacity  Mounted on
> /dev/md2      372M    100M    242M    29%    /tmp1
> Filesystem    Size    Used   Avail Capacity  Mounted on
> /dev/md2      372M    112M    229M    33%    /tmp1
> Filesystem    Size    Used   Avail Capacity  Mounted on
> /dev/md2      372M    125M    217M    36%    /tmp1
> .....
> 
> The "Used" space grows quickly until "No space available".
> After umount /var/tmp1 the space is back again.
> 
> I see the same behaviour when md is replaced by tmpfs.
> 
> I have tried the patch from commit 317936 (V11) without success.
> The problem exists on i386 and amd64. In production (V8) I use this
> double mount construction for jail communication without any problems.

Can you try the _latest_ stable/11 ?  It is enough to only install the
kernel, keep the userspace.

If it does not help, perhaps you will need the -o nocache mount option.