[Bug 209112] /usr/sbin/jail jails fail to launch with possible race when jails mount common dir with nullfs

[bugzilla-noreply at freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=209112

djn at araxis.com changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |djn at araxis.com

--- Comment #4 from djn at araxis.com ---
I am also seeing this issue with FreeBSD 10.3-Release-p4 – I have six jails,
and only two or three start on boot (two fail due to a dependency on another
jail that fails to start). I encounter no problems when starting the jails
post-boot.

I added logging to /etc/rc.d/jail in a similar manner to the original poster,
and see this output:

----
mount_nullfs: /jails/pgsql/data: mount_nullfs: /jails/hg/data: Operation not
supported by device
Operation not supported by device
jail: pgsql: /sbin/mount -t nullfs -o rw /data/pgsql /jails/pgsql/data: failed
jail: hg: /sbin/mount -t nullfs -o rw /data/hg /jails/hg/data: failed
jail: support: skipped
jail: logic: skipped
www-bhs-0: created
backup-bhs-0: created

...
----

Here is my jail.conf:

----
# START BLOCK: Araxis jail defaults -- DO NOT EDIT
exec.start = "/bin/sh /etc/rc";
exec.stop = "/bin/sh /etc/rc.shutdown";
exec.clean;
mount.devfs;
mount.fdescfs;
allow.noset_hostname;
host = new;

# temporarily allow during development/testing
allow.raw_sockets;
# END BLOCK: Araxis jail defaults
# START BLOCK: Araxis jail settings for jail www-bhs-0.araxis.net -- DO NOT
EDIT
www-bhs-0 {
  mount.fstab = "/jails/.fstabs/www-bhs-0";
  path = "/jails/www-bhs-0";
  host.hostname = www-bhs-0.araxis.net;
  ip4.addr = "lo1|10.11.11.2/32";
  ip6.addr = "ix0|2607:5300:60:9b9c::2/64";
}
# END BLOCK: Araxis jail settings for jail www-bhs-0.araxis.net
# START BLOCK: Araxis jail settings for jail backup-bhs-0.araxis.net -- DO NOT
EDIT
backup-bhs-0 {
  mount.fstab = "/jails/.fstabs/backup-bhs-0";
  path = "/jails/backup-bhs-0";
  host.hostname = backup-bhs-0.araxis.net;
  ip4.addr = "lo2|10.12.12.8/32";
  enforce_statfs = 1;
  allow.mount;
  allow.mount.zfs;
  exec.poststart = "/sbin/zfs set jailed=on zroot/bkup && /sbin/zfs jail
${name} zroot/bkup && jexec ${name} /sbin/zfs mount -a";
  exec.prestop = "/sbin/zfs unjail ${name} zroot/bkup && /sbin/zfs set
jailed=off zroot/bkup";
}
# END BLOCK: Araxis jail settings for jail backup-bhs-0.araxis.net
# START BLOCK: Araxis jail settings for jail pgsql.araxis.net -- DO NOT EDIT
pgsql {
  mount.fstab = "/jails/.fstabs/pgsql";
  path = "/jails/pgsql";
  host.hostname = pgsql.araxis.net;
  ip4.addr = "lo2|10.12.12.4/32";
  allow.sysvipc;
}
# END BLOCK: Araxis jail settings for jail pgsql.araxis.net
# START BLOCK: Araxis jail settings for jail support.araxis.net -- DO NOT EDIT
support {
  mount.fstab = "/jails/.fstabs/support";
  path = "/jails/support";
  host.hostname = support.araxis.net;
  ip4.addr = "lo2|10.12.12.5/32";
  depend = pgsql;
}
# END BLOCK: Araxis jail settings for jail support.araxis.net
# START BLOCK: Araxis jail settings for jail logic.araxis.com -- DO NOT EDIT
logic {
  mount.fstab = "/jails/.fstabs/logic";
  path = "/jails/logic";
  host.hostname = logic.araxis.com;
  ip4.addr = "lo2|10.12.12.6/32";
  depend = pgsql;
}
# END BLOCK: Araxis jail settings for jail logic.araxis.com
# START BLOCK: Araxis jail settings for jail hg.araxis.net -- DO NOT EDIT
hg {
  mount.fstab = "/jails/.fstabs/hg";
  path = "/jails/hg";
  host.hostname = hg.araxis.net;
  ip4.addr = "lo2|10.12.12.3/32";
}
# END BLOCK: Araxis jail settings for jail hg.araxis.net
----

The various fstab files are nearly identical. Here’s the one
(/jails/.fstabs/pgsql) for the pgsql jail, which always fails to start on boot:

----
# Device    Mountpoint    FStype    Options   Dump    Pass#
/data/pgsql   /jails/pgsql/data    nullfs    rw    0   0
----

-- 
You are receiving this mail because:
You are the assignee for the bug.