mount_unionfs for jails

Nejc S(koberne nejc at
Thu Sep 25 20:33:17 UTC 2008


> Here is what I am trying to do:
> mount_unionfs -o below /usr/jails/basejail /usr/jail/jail1
> after I do that I edit /usr/jail/jail1/etc/rc.conf and add the appropriate
> entries to the host system rc.conf, but when I start the jail it starts
> using the settings from /usr/jails/basejail.

I have the same setup and it works for me.

> Is my mount_unionfs syntax wrong, is this a bug in unionfs (man page says
> unionfs is broken, but doesn't specify how its broken) or is this expected
> behavior?

It shouldn't be wrong. I have this in my fstab:

/jail/base   /jail/spl/nejcspl       unionfs rw,noatime,below     0       0

(noatime option is completely optional, of course.)

But, if I were you, I would update the RELEASE to STABLE. This will also fix
some bugs in unionfs. However at least some other bugs still aren't fixed in 7-STABLE
to this day (most notably the socket bug, which prevents mysql from running in a jail
and writing socket file to /tmp/mysql.sock), so we had to MFC the patch from
HEAD manually. If you need the patch, let me know.

However, I don't suggest running jails on top of unionfs where you need
decent stability (i.e. in production). I am writing thesis at the moment which
also covers this topic. We also stumbled upon these issues:

- socket file bug, mentioned before, still present in 7-STABLE, no ideas
  when it will be MFCed;

- "mv" bug (see freebsd-fs archives for August 2008, me and my friend posted
  a few posts there) which causes troubles when moving directories (files
  would appear as gone and then reappear again) which exist or don't exist
  on lower and upper levels;

- another "mv" bug which I discovered yesterday and seems to be very strange
  and hard to replay - I didn't even mess with the lower level, it seems that
  also just the upper layer can behave strangely sometimes (erros like
  "mv: invalid argument" when simply trying to move a big (>10 GB) directory -
  the error was gone after I restarted the jail (i.e. also remounting the

- strange behaviour of some applications (apache in my case) not "seeing" the
  lower layer (/etc/hosts most notably) - we had to do "touch" (and then copy
  to all jails on change) on files we _really_ need to be visible. However,
  after we "fscked" our partition with unionfs directories, we weren't able
  to reproduce this error;

- UFS filesystem would get to inconsistent state (we don't know exactly when)
  so some commands would behave strangely and fsck (see above) is needed in
  single user mode;

- _most notably_: there hasn't been a single reply to our unionfs related
  problem reports and posts to freebsd-fs list. So I guess that people who
  are in charge for unionfs in FreeBSD aren't really responsive and that
  the future of unionfs in FreeBSD isn't really bright. It's a pity, though,
  since this is a very useful feature, especially for jailed systems. However,
  hope remains, that things will be fixed at least in 8.0 if not in 7.1.

So, you can see that there are (still) many issues with unionfs on FreeBSD.
Please let me know if you are able to solve your problem. Or else we can make
this list a little longer. :)


More information about the freebsd-questions mailing list