vnet jail and ipfw/nat on host - keep-state problem?

[Peter Toth]

Dear Joe Barbish (alias  fbsd8 at a1poweruser.com),

When you going to stop trolling the FreeBSD mailing list and spread
disinformation? For anyone interested please check this mail thread on who
fbsd8 really is:
http://lists.freebsd.org/pipermail/freebsd-jail//2013-March/002147.html

Very telling isn't it!

People come to this place to learn, share information, help out other folks
and most importantly to have a constructive debate! (obviously some would
rather divert this effort)

The PR number's mentioned are mostly outdated from the 8.x and 9.x series -
some of them are completely irrelevant (like ACPI) or for a i386 system.
Beyond this I am categorically refusing to waste any energy and time on
answering any trolling/diversion attempts by Joe Barbish.
Most importantly I encourage anyone avoiding his dubious Qjail project by
far - for details please check the link above.

I am not going to burn time on dissecting each PR one-by-one but rather
share my experience with VNET.

Over the last year and a half have deployed numerous production systems
based on amd64 10-RELEASE with VNET enabled and PF running on the host.
Encountered 0 instability issues! Details on how to do this are here:
http://iocage.readthedocs.org/en/latest/real-world.html

As I mentioned before IPFW works in a jail and PF only works on the host.

Back to the original issue though, Peter could you please share your IPFW
config with me (maybe just send it directly to me), would be very
interested to get it going in my lab setup and add a howto page to share
this with others.

Cheers,
Peter









On Sat, Jul 12, 2014 at 1:16 PM, Fbsd8 <fbsd8 at a1poweruser.com> wrote:

> Peter Toth wrote:
>
>  On Sat, Jul 12, 2014 at 1:33 AM, Fbsd8 <fbsd8 at a1poweruser.com <mailto:
>> fbsd8 at a1poweruser.com>> wrote:
>>
>>     Peter Toth wrote:
>>
>>         Have not used natd with IPFW much as always preferred PF to do
>>         everything
>>         on the host.
>>
>>         I have only a wild guess - the "me" keyword in IPFW is
>>         substituted only to
>>         the host's IPs known to itself.
>>         The host's IPFW firewall most likely doesn't know anything about
>> IPs
>>         assigned to vnet interfaces inside the jail.
>>
>>         Vnet jails behave more like separate physical hosts.
>>
>>         Internet ---> [host] ------- (10.0.10.0 LAN) ------> [vnet jail]
>>
>>         The PF issue inside a jail is a separate problem, PF is not fully
>>         VIMAGE/VNET aware as far as I know.
>>
>>         Can someone comment on these or correct me?
>>
>>         P
>>
>>
>>
>>         On Fri, Jul 11, 2014 at 7:11 PM, Peter Ross
>>         <Peter.Ross at alumni.tu-berlin.__de
>>         <mailto:Peter.Ross at alumni.tu-berlin.de>>
>>
>>         wrote:
>>
>>             On Thu, 10 Jul 2014, Peter Toth wrote:
>>
>>              Hi Peter,
>>
>>                 Try to make these changes:
>>
>>                 net.inet.ip.forwarding=1       # Enable IP forwarding
>>                 between interfaces
>>                 net.link.bridge.pfil_onlyip=0  # Only pass IP packets
>>                 when pfil is enabled
>>                 net.link.bridge.pfil_bridge=0  # Packet filter on the
>>                 bridge interface
>>                 net.link.bridge.pfil_member=0  # Packet filter on the
>>                 member interface
>>
>>                 You can find some info
>>                 here
>>                 http://iocage.readthedocs.org/
>> __en/latest/help-no-internet.__html
>>
>>                 <http://iocage.readthedocs.org/en/latest/help-no-
>> internet.html>
>>
>>                 I've had these issues before with PF and IPFW, by
>>                 default these will be
>>                 filtering on your bridge and member interfaces.
>>
>>             Thanks. It did not change anything.
>>
>>             Now, inside_ the jail I run "ipfw allow ip from any to any".
>>
>>             This on the host system:
>>
>>             01000 check-state
>>             01100 allow tcp from any to any established
>>             01200 allow ip from any to any frag
>>             00100 divert 8668 ip4 from any to any via age0
>>             03100 allow udp from any to 10.0.10.1 dst-port 53 keep-state
>>             03200 allow udp from any to me dst-port 53 keep-state
>>
>>             (with natd redirecting "redirect_port udp 10.0.10.1:53
>>             <http://10.0.10.1:53> external.ip:53")
>>
>>
>>             If I add
>>
>>             03300 allow udp from me 53 to any
>>
>>             it works..
>>
>>             So it makes me think check-state isn't usable - because
>>
>>             03200 allow udp from any to me dst-port 53 keep-state
>>
>>             should cover the returning packets.
>>
>>             I played with your parameters but it did not help. But
>>             thanks for the idea.
>>
>>             Here again the setup:
>>
>>             Internet->age0(host interface with natd and external IP)
>>             ->bridge10(10.0.10.254)->__epair1a
>>
>>             ->epair1b(10.0.10.1 in bind vnet jail)
>>
>>             I wonder what kind of restrictions exist with vnet.. it does
>>             not seem to
>>             work _exactly_ as a "real" network stack (the issues with pf
>>             inside the
>>             jail let me think of it too)
>>
>>             Did I find a restriction, a bug - or just that I've got it
>>             wrong?
>>
>>             Regards
>>             Peter
>>
>>
>>     Any firewall function that runs in the kernel will not function
>>     inside of a vnet/vimage jail.
>>
>>
>>
>> This sounds a bit vague, can you please explain in more detail what you
>> meant by this?
>>
>> IPFW works inside a vnet jail - You can manage per jail firewall
>> instances without any issues.
>>
>> The only firewall which cannot function inside a jail (yet) is PF.
>>
>> P
>>
>>
>>
> You are incorrect.
> Here is a list of some of the vnet/vimage outstanding PR's
>
> 143808, 147950, 148155, 152148, 160496, 160541, 161094, 164763, 165252,
> 176112, 176929, 178480, 178482, 179264, 182350, 185092, 188010, 191468
>
>
>
>
>
>
>