[Bug 238326] Kernel crash on jail stop (VIMAGE/VNET)
bugzilla-noreply at freebsd.org
bugzilla-noreply at freebsd.org
Tue Jun 4 21:05:28 UTC 2019
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=238326
Bug ID: 238326
Summary: Kernel crash on jail stop (VIMAGE/VNET)
Product: Base System
Version: 12.0-RELEASE
Hardware: Any
OS: Any
Status: New
Severity: Affects Only Me
Priority: ---
Component: kern
Assignee: bugs at FreeBSD.org
Reporter: paul.le.gauret at gmail.com
Jails managed through the native jails tools with the attached jail.conf file
(so called "thin jails", VNET). I can reliably crash the system by issuing a
"service jail restart". The crash happens when jails are being stopped.
[root at beastie ~]# uname -a
FreeBSD beastie 12.0-RELEASE-p4 FreeBSD 12.0-RELEASE-p4 GENERIC amd64
[root at beastie ~]# service jail restart
Stopping jails: dns db ldap web nextcloud imap smtp testpacket_write_wait:
Connection to xxx.xxx.xxx.xxx port 22: Broken pipe
As a workaround I can avoid the crash by inserting a
exec.poststop = "sleep 2";
statement in jail.conf. 1 second was not enough to avoid the crash.
Also worth noting that the issue does not appear if I totally disable
networking, hence my guess this is somehow VIMAGE/VNET related.
I've managed to obtain a core dump but my attempts at debugging didn't go far
as I'm not experienced with this. Of course happy to do more debugging if this
can help identifying the issue. The system is not yet live so I can pretty much
try anything on it.
[root at beastie /var/crash]# kgdb /boot/kernel/kernel /var/crash/vmcore.0
GNU gdb (GDB) 8.2.1 [GDB v8.2.1 for FreeBSD]
Copyright (C) 2018 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-portbld-freebsd12.0".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /boot/kernel/kernel...(no debugging symbols found)...done.
0xffffffff80bcd0bd in sched_switch ()
(kgdb) bt
#0 0xffffffff80bcd0bd in sched_switch ()
#1 0xffffffff80ba6de1 in mi_switch ()
#2 0xffffffff80bf554c in sleepq_wait ()
#3 0xffffffff80ba6817 in _sleep ()
#4 0xffffffff80bfae71 in taskqueue_thread_loop ()
#5 0xffffffff80b5bf33 in fork_exit ()
#6 <signal handler called>
My jail.conf file below. Note that the crash happens regardless of whether the
jails with special permissions (db, builder) are configured. I also tried
tweaking the mount/umount order without success.
$release="12.0-RELEASE"; # Release used to create the jail if not already
existing
host.hostname = "${name}";
path = "/jails/${name}";
exec.consolelog = "/var/log/jail.${name}.console.log";
vnet = "new";
vnet.interface = "epair${jailnum}b";
### Create the jail if not already exsisting ###
exec.prestart = "if [ ! -d "/jails/THINJAILS/${name}" ]; then zfs clone
zroot/jails/TEMPLATES/skeleton-${release}@skeleton
zroot/jails/THINJAILS/${name}; fi";
exec.prestart += "if [ ! -d "/jails/${name}" ]; then mkdir /jails/${name}; fi";
### Mount filesystems for shared base (RO) and individual jail skeleton (RW)
exec.prestart += "mount_nullfs -o ro /jails/TEMPLATES/base-${release}
/jails/${name}";
exec.prestart += "mount_nullfs -o rw /jails/THINJAILS/${name}/etc
/jails/${name}/etc";
exec.prestart += "mount_nullfs -o rw /jails/THINJAILS/${name}/home
/jails/${name}/home";
exec.prestart += "mount_nullfs -o rw /jails/THINJAILS/${name}/root
/jails/${name}/root";
exec.prestart += "mount_nullfs -o rw /jails/THINJAILS/${name}/tmp
/jails/${name}/tmp";
exec.prestart += "mount_nullfs -o rw /jails/THINJAILS/${name}/var
/jails/${name}/var";
exec.prestart += "mount_nullfs -o rw /jails/THINJAILS/${name}/usr/local
/jails/${name}/usr/local";
### Mount /usr/ports from hosts
exec.prestart += "mount_nullfs -o ro /usr/ports /jails/${name}/usr/ports";
### Mount data filesystem if exists
exec.prestart += "if [ -d "/jails/DATA/${name}" ]; then mount_nullfs -o rw
/jails/DATA/${name} /jails/${name}/data; fi";
### Mount devfs with the default ruleset (11) now that the jail filesystems are
mounted
exec.prestart += "mount -t devfs -o ruleset=4 devfs /jails/${name}/dev";
### Create an ethernet pair and add to bridge0 (internal bridge)
exec.prestart += "ifconfig epair${jailnum} create up";
exec.prestart += "ifconfig epair${jailnum}a description '${name} - host
interface'";
exec.prestart += "ifconfig epair${jailnum}b description '${name} - jail
interface'";
exec.prestart += "ifconfig bridge0 addm epair${jailnum}a";
exec.start = "ifconfig epair${jailnum}b inet 192.168.1.${jailnum}/24";
exec.start += "ifconfig epair${jailnum}b inet6
xxx:xxx:xxx:xxx::${jailnum}:1/64";
exec.start += "route add default 192.168.1.1";
exec.start += "route -6 add default xxx:xxx:xxx:xxx::1:1";
### Proceed with boot through rc
exec.start += "/bin/sh /etc/rc";
### Shutdown the jail
exec.stop = "/bin/sh /etc/rc.shutdown";
### Give the jail a couple of seconds to shutdown (avoid issues unmounting
filesystems)
#exec.poststop = "sleep 2";
### Destroy jail network interface
exec.poststop += "ifconfig epair${jailnum}a destroy";
### Unmount filesystems
exec.poststop += "if [ -d "/jails/DATA/${name}" ]; then umount
/jails/${name}/data; fi";
exec.poststop += "umount -f /jails/${name}/dev";
exec.poststop += "umount -f /jails/${name}/usr/ports";
exec.poststop += "umount -f /jails/${name}/etc";
exec.poststop += "umount -f /jails/${name}/home";
exec.poststop += "umount -f /jails/${name}/root";
exec.poststop += "umount -f /jails/${name}/tmp";
exec.poststop += "umount -f /jails/${name}/var";
exec.poststop += "umount -f /jails/${name}/usr/local";
exec.poststop += "umount -f /jails/${name}";
dns { $jailnum=2; };
db {
$jailnum=3;
allow.sysvipc;
};
ldap { $jailnum=4; };
web { $jailnum=5; };
nextcloud { $jailnum=6; };
imap { $jailnum=7; };
smtp { $jailnum=8; };
test { $jailnum=98; };
builder {
$jailnum=99;
enforce_statfs=0;
allow.mount;
allow.mount.nullfs;
allow.mount.tmpfs;
allow.mount.devfs;
allow.chflags;
};
--
You are receiving this mail because:
You are the assignee for the bug.
More information about the freebsd-bugs
mailing list