umounting md backed jail filesystems - busy

[Ian Downes]

Hi everyone,

I'm trying to cleanup after shutting down some jails but I'm getting device
busy errors when trying to umount some of the filesystems.

More specifically, I've got an ephemeral zfs filesystem that serves as the root
of the jail. On '/etc/rc.d/jail stop' the jail stops cleanly but when I try to
destroy the zfs filesystem the initial umount fails, claiming it's busy. This
happens everytime.

I can't for the life of me work out who's tying it up. I've tried fstat, lsof
and fuser but nothing is reported as active! No processes, no active files.

Details:

$ uname -a
FreeBSD XXX.com 8.2-RELEASE FreeBSD 8.2-RELEASE #0: Tue May 31 19:05:32 UTC 2011     root at XXX.com:/usr/obj/usr/src/sys/XENHVM  amd64

'data' is a md backed zpool

$ mount | grep
data/path/to/jail/root on /path/to/jail/root (zfs, local)

$ fstat -f /path/to/jail/root
USER     CMD          PID   FD MOUNT      INUM MODE         SZ|DV R/W

$ unmount /path/to/jail/root
cannot unmount '/path/to/jail/root': Device busy

Some time later, measured in minutes, something frees up and I can
umount/destroy the filesystem ok.

Can anyone offer some suggestions on what it could be or other ways to
determine what's going on?

Thanks,

Ian