vnet jail with ipfw having logging problem

[Joe]

Anders Hagman wrote:
> Hi
> 
> 2 maj 2013 kl. 07:42 skrev Ian Smith <smithi at nimnet.asn.au>:
> 
>> On Wed, 1 May 2013 17:43:03 -0400, Joe wrote:
>>>>> I have ipfw running inside of a vnet jail on a 9.1-RELEASE host using
>>>> the
>>>>> jail(8) definition statements for starting and stopping the vnet jail.
>>>> As a
>>>>> side note non-vnet jails are working as expected.
>>>>>> The host is running a custom kernel with modules and with
>>>>> options VIMAGE
>>>>> nooptions SCTP
>>>>> options IPFIREWALL
>>>>> options IPFIREWALL_VERBOSE
>>>>> options IPFIREWALL_VERBOSE_LIMIT=10
>> Please maintain attributions for the archives.  I wrote:
>>
>>>> What steps have you taken during testing to override this ridiculously low
>>>> limit on logging?  Otherwise, after e.g. just 5 pings and 5 ping responses
>>>> are logged, all logging ceases until issuing 'ipfw resetlog'.
>>> /usr/src/sys/conf/NOTES says IPFIREWALL_VERBOSE_LIMIT; limits the number of
>>> times a matching entry can be logged. Says nothing about this limit being the
>>> maximum number of log records allowed after which the log file is closed for
>>> business. Are you saying the /usr/src/sys/conf/NOTES info is no longer true?
>> You showed one (1) 'log' rule for each of the host's and jail's ruleset. 
>> Once that one rule has been logged 'logamount' times (default as per 
>> NOTES is 100, but in your case is 10) then logging for THAT rule stops, 
>> therefore with only one 'log' rule, ALL logging stops.  Understand?
>>
>> If you take the time to properly study the correct reference, ipfw(8), 
>> all of this will become clear.  See especially section SYSCTL VARIABLES, 
>> and read thoroughly 'log [logamount number]', at the very least.  Ignore 
>> the Handbook section on ipfw, it's full of errors and misunderstandings.
>>
>>> Without IPFIREWALL_VERBOSE and IPFIREWALL_VERBOSE_LIMIT where does the logged
>>> packets get written to? /var/log/security
>> See above.  Both of these options merely set defaults for the sysctls.
>>
>>> I have not used ipfw since it's ipfw2 rewrite so my knowledge is dated.
>> Indeed it is; that's a very long time ago.
>>
>>>>> options IPFIREWALL_DEFAULT_TO_ACCEPT
>>>>> options IPFIREWALL_IPDIVERT
>>>> You'd likely do better using in-kernel NAT; natd doesn't get much love.
>>>>
>>> I kept getting kernel compile errors using "options IPFIREWALL_NAT". I
>>> thought the error was caused by vimage. Now I know "options LIBALIAS" is
>>> required. Could not find info on internet search for IPFIREWALL_NAT with
>>> vimage kernel.
>> Apart from FIREWALL_FORWARD (not even that in 10.x), none of that needs 
>> to be in the kernel, it's all loadable as modules; see /etc/rc.d/ipfw.
>>
>> If you're doing NAT in the vimage jail, you must have at least two 
>> interfaces assigned to the jail.  Care to show your config for that?
>>
>>> Do you have first hand experience getting "ipfw kernel nat" to work in a
>>> vimage jail or having logging work on the host and within the vnet jail?
>> No, but I have just on 15 years experience managing ipfw firewalls :)
> 
> When you are new at things you do mistakes, remember.
> 
> To try to answer Joes question:
> 
> You don't need to compile anything into the kernel regarding ipfw.
> 
> Just load the ipfw module in the host system with:
> 
>   kldload ipfw
> 
> By default a deny all rule is added, so add a allow rule to the host system.
> 
>   ipfw add 10 allow ip from any to any
> 
> To log things you change the sysctl value net.inet.ip.fw.verbose to 1
> 
>   sysctl net.inet.ip.fw.verbose=1
> 
> If you keep net.inet.ip.fw.verbose_limit=0 you don't have a log limit, and for tests thats fine.
> 
> log in to the jail system. Change the  sysctl value net.inet.ip.fw.verbose to 1
> 
>   sysctl net.inet.ip.fw.verbose=1
> 
> Add a logging firewall rule
> 
>   ipfw add 10 allow log ip from any to any
> 
> Do a ping to an external system.
> Look inside /var/log/security in the jail system and its empty.
> Go to the main host and look at the /var/log/security file and you will find log entries.
> 
> I can confirm Joes bug. I don't have a log rule in the main host but still get log messages.
> All log messages are from the log rule in the jail system.
> 
> System used: 9.1-RELEASE-p2
> 
> BR
> /Anders

Thank you Anders, your reply was direct and to the point.

Lets talk about this bug. The console.log parameter creates a log file 
in the hosts /var/log directory for each jail. I would think the ipfw 
log file should behave the same way. IE: the host ipfw log should be 
going to the hosts /var/log/security file with each ipfw jail creating 
it's own /var/log/jailname.security file on the host and not create a 
/var/log/security file in the jails filesystem. I searched the PR 
database for any PR's with vnet or vimage and ipfw logging and came up 
with no hits. Should I submit a PR about this problem?

I tested doing a kldload ipfw and fall into the default deny problem.
Is there a sysctl to flip the default deny to default accept?

Thanks
Joe