Vimage Jail kernel crashed

[KIRIYAMA Kazuhiko]

Hi,

At Sat, 4 May 2013 15:07:11 +0300,
Mikolaj Golub wrote:
> 
> On Sat, May 04, 2013 at 02:52:23PM +0900, KIRIYAMA Kazuhiko wrote:
> 
> > May  4 11:19:46 xxxxxx kernel: Fatal trap 12: page fault while in kernel mode
> > May  4 11:19:46 xxxxxx kernel: cpuid = 2; apic id = 02
> > May  4 11:19:46 xxxxxx kernel: fault virtual address	= 0x7818c3798
> > May  4 11:19:46 xxxxxx kernel: fault code		= supervisor write data, page not present
> > May  4 11:19:46 xxxxxx kernel: instruction pointer	= 0x20:0xffffffff8162c19e
> > May  4 11:19:46 xxxxxx kernel: stack pointer	        = 0x28:0xffffff8121b22860
> > May  4 11:19:46 xxxxxx kernel: frame pointer	        = 0x28:0xffffff8121b22870
> > May  4 11:19:46 xxxxxx kernel: code segment		= base 0x0, limit 0xfffff, type 0x1b
> > May  4 11:19:46 xxxxxx kernel: = DPL 0, pres 1, long 1, def32 0, gran 1
> > May  4 11:19:46 xxxxxx kernel: processor eflags	= interrupt enabled, resume, IOPL = 0
> > May  4 11:19:46 xxxxxx kernel: current process		= 15360 (ifconfig)
> > May  4 11:19:46 xxxxxx kernel: trap number		= 12
> > May  4 11:19:46 xxxxxx kernel: panic: page fault
> > May  4 11:19:46 xxxxxx kernel: cpuid = 2
> > May  4 11:19:46 xxxxxx kernel: KDB: stack backtrace:
> > May  4 11:19:46 xxxxxx kernel: #0 0xffffffff80923446 at kdb_backtrace+0x66
> > May  4 11:19:46 xxxxxx kernel: #1 0xffffffff808ed0be at panic+0x1ce
> > May  4 11:19:46 xxxxxx kernel: #2 0xffffffff80c7e330 at trap_fatal+0x290
> > May  4 11:19:46 xxxxxx kernel: #3 0xffffffff80c7e668 at trap_pfault+0x1e8
> > May  4 11:19:46 xxxxxx kernel: #4 0xffffffff80c7ec6e at trap+0x3be
> > May  4 11:19:46 xxxxxx kernel: #5 0xffffffff80c682ef at calltrap+0x8
> > May  4 11:19:46 xxxxxx kernel: #6 0xffffffff8162c76d at pfi_change_group_event+0x4d
> > May  4 11:19:46 xxxxxx kernel: #7 0xffffffff809a0d3b at if_delgroup+0x38b
> > May  4 11:19:46 xxxxxx kernel: #8 0xffffffff809a7846 at if_clone_destroyif+0x136
> > May  4 11:19:46 xxxxxx kernel: #9 0xffffffff809a831a at if_clone_destroy+0x17a
> > May  4 11:19:46 xxxxxx kernel: #10 0xffffffff809a5892 at ifioctl+0x482
> > May  4 11:19:46 xxxxxx kernel: #11 0xffffffff80934ef6 at kern_ioctl+0x106
> > May  4 11:19:46 xxxxxx kernel: #12 0xffffffff8093513d at sys_ioctl+0xfd
> > May  4 11:19:46 xxxxxx kernel: #13 0xffffffff80c7dc10 at amd64_syscall+0x540
> > May  4 11:19:46 xxxxxx kernel: #14 0xffffffff80c685d7 at Xfast_syscall+0xf7
> 
> It looks like it crashed when referring vnet that had already been
> destroyed, in pfi_change_group_event hook.
> 
> > Is there any suggestions? 
> 
> VIMAGE+pf support is fragile. If it works for someone it is rather by
> accident. I expect replacing pf with ipfw_nat or natd will give better
> results.

I've been used natd before using VIMAGE Jail private network. And I tried to
use with ipfw+natd like this:

/etc/rc.conf:
defaultrouter="X.X.X.X"
hostname="hoge.org"
ifconfig_em0="inet X.X.X.X netmask 255.255.255.240"
cloned_interfaces="bridge0"
ifconfig_bridge0="inet 192.168.1.254 netmask 255.255.255.0 up"
jail_v2_enable="YES"
ezjail_enable="YES"
gateway_enable="YES"
firewall_enable="YES"
firewall_type="OPEN"
natd_enable="YES"
natd_interface="em0"
natd_flags="-config /etc/natd.cf"
keymap="jp.106"
sshd_enable="YES"
ntpd_enable="YES"
sendmail_enable="NONE"
dumpdev="NO"

/etc/natd.cf:
log			no
deny_incoming		yes
use_sockets		no
same_ports		yes
verbose			no
port			8668
interface		em0
unregistered_only	yes
redirect_port		tcp 192.168.1.1:domain	domain
redirect_port		udp 192.168.1.1:domain	domain
redirect_port		tcp 192.168.1.3:cvspserver	cvspserver
redirect_port		udp 192.168.1.4:cvspserver	cvspserver
redirect_port		tcp 192.168.1.4:smtp	smtp
redirect_port		tcp 192.168.1.4:pop3	pop3
redirect_port		tcp 192.168.1.5:http	http
redirect_port		tcp 192.168.1.254:ABCDE	ABCDE
redirect_port		tcp 192.168.1.1:ssh	ABBCA
redirect_port		tcp 192.168.1.2:ssh	ABBCB
redirect_port		tcp 192.168.1.3:ssh	ABBCC
redirect_port		tcp 192.168.1.4:ssh	ABBCD
redirect_port		tcp 192.168.1.5:ssh	ABBCE

/boot/loader.conf:
ipdivert_load="YES"
ipfw_load="YES"

But all packet redirection were neglected except 192.168.1.254 namely
firewall itself ;)

> If you still prefer pf, you may try destroying epair interface before
> destroying vnet, e.g. using prestop rc.d/jail hooks instead of
> poststop, if it is possible.

In particular, execute following sequence?

# ifconfig epairXa destroy
# ifconfig bridge0 deletem epairXa

> -- 
> Mikolaj Golub
>