vnet jail with ipfw having logging problem

[Joe]

I have ipfw running inside of a vnet jail on a 9.1-RELEASE host using 
the jail(8) definition statements for starting and stopping the vnet 
jail. As a side note non-vnet jails are working as expected.

The host is running a custom kernel with modules and with
options VIMAGE
nooptions SCTP
options IPFIREWALL
options IPFIREWALL_VERBOSE
options IPFIREWALL_VERBOSE_LIMIT=10
options IPFIREWALL_DEFAULT_TO_ACCEPT
options IPFIREWALL_IPDIVERT
options IPFIREWALL_FORWARD

compiled in.


The host is also running ipfw from its rc.conf file.

Both the vnet jail and the host, have ipfw rules, for logging everything.

Host
ipfw -q add 010 allow all from any to any via lo0
ipfw -q add 010 allow log all from any to any via rl0


vnet jail
# pass the vnet epairXb to the jail rule.
if [ -e /etc/epair ]; then
    pif=`cat "/etc/epair"`
else
    pif="lo0"
fi
ipfw -q add 010 allow all from any to any via lo0
ipfw -q add 010 allow log all from any to any via $pif

I work around the (nojail keyword problem with the rc.d startup scripts)
by manually issuing service netif start, service routing start, service 
ipfw start commands, after the exec.start="/bin/sh /etc/rc" is run and 
issue them in reverse order before the
exec.stop="/bin/sh /etc/rc.shutdown" is executed.

After booting the system I can ping the internet from the host and see 
the rule counter increase using this command "ipfw -a list". I also see 
the ping packets logged in the hosts /var/log/security file.

After the vnet jail is started I see a empty /var/log/security file
inside of the vnet jail that never gets populated.

But in the hosts /var/log/security file I see log messages from the
vnet jail. I would expect to see the vnet jail log message interspersed 
with the host logging messages as the jails packets pass through the 
hosts ipfw firewall.

But I only see the vnet jails ipfw logging messages in the hosts 
/var/log/security file from that point on.

I can stop the vnet jail and restart it and the ipfw logged messages 
continue to populate the hosts security file.

With the vnet jail stopped, I issue ping from host to internet and get 
100% packets replied message. The ipfw -a list command from the host 
shows the rule count has increased, but there are no host packets logged 
to the hosts security file.

Rebooting the host is the only way to get the host to log ipfw packets 
and this only works until the vent jail starts after which time the host 
no longer logs packets.

I can comment out the firewall statements in the hosts rc.conf and 
reboot the host without ipfw, but since ipfw is compiled into the kernel 
ipfw is really running on the host with the allow all default and the 
started vmet/ipfw jail still populates the hosts security file.

I'm thinking this is a bug.

I would like conformation of this problem.

Or maybe someone has some other (nojail keyword work around method) that 
results in the vnet/ipfw jail logging to the jails security file and the 
hosts ipfw logging to it's security file that they would share with me.


Thanks
Joe