Changes to ipfw in 8.1

Spil Oss spil.oss at gmail.com
Thu Jul 22 18:24:59 UTC 2010


On Thu, Jul 22, 2010 at 12:16 PM, Ian Smith <smithi at nimnet.asn.au> wrote:
>
> This points to a number of issues, not least the effect of people being
> led to using the currently dreadful IPFW section of the handbook, which
> contains so many conceptual and factual errors that I really don't know
> where to begin on a solution .. so I won't discuss that further except
> as it relates to the actual ruleset you're using.

I'm not a network admin, and happy that it does what I want at all. My
concern is mainly that an upgrade to 8.1 will break many systems.

> In your just-filed PR http://www.freebsd.org/cgi/query-pr.cgi?pr=148827
> you say your ruleset is based on '30.6.5.7 An Example NAT and Stateful
> Ruleset', so I'm assuming it's broadly based on example #2 there.
>
>  > On Wed, Jul 21, 2010 at 9:08 PM, Spil Oss <spil.oss at gmail.com> wrote:
>  > > Hi Sergey,
>  > >
>  > > Has the change from ip to ip4 solved the problem for you? The
>  > > documentation states that proto 'ip' is the same as 'all' "Matches any
>  > > packet."
>  > >
>  > > Rule # 60
>  > >     $cmd 060 skipto 1000 ip6 from any to any
>  > > will have already skipped to the ipv6 rules block thus proto 'ip'
>  > > should always match remaining packets.
>
> You don't show any rules beyond number 510, so we must take it on faith
> that you're handling your ip6 traffic appropriately.  I don't know much
> about it, and will be guided by the IPv6 rules lately in rc.firewall.
>

For completeness
########################  IPv6 rules   ##################

# Allow the uptime pings from SixXS
$cmd 1000 allow icmp6 from 2001:dead:beef:c10::1 to
2001:dead:beef:c10::2 in via $pif6
$cmd 1000 allow icmp6 from me6 to 2001:dead:beef:c10::1 out via $pif6

# Open up all IPv6
#$cmd 006 allow ip6 from any to any via $pif6

$cmd 1100 check-state

# Authorized outbound packets
$cmd 1160 allow ip6 from any to any out via $pif6 $ks
$cmd 1161 allow udp from any to any out via $pif6 $ks
$cmd 1162 allow icmp6 from any to any out via $pif6 $ks

# Authorized inbound packets
$cmd 1450 allow tcp from any to $ext6 dst-port ssh in via $pif6

$cmd 1999 reject log all from any to any in via $pif6
$cmd 1999 reject log all from any to any out via $pif6
$cmd 1999 reject log ip from any to any

######################## end of rules  ##################
> However if natd is barfing on being passed ip6 packets, that should be
> fixed in natd, which should do nothing with packets it doesn't care
> about - as it does with ip4 packets not eligible for NAT translation -
> except to re-enter the firewall at the next higher-numbered rule, which
> of course relies on sysctl net.inet.ip.fw.one_pass being set to 0.

That may be an issue....
net.inet.ip.fw.one_pass: 1

> Either that or ipfw should explicitly decline to divert non-ip4 traffic.
> I don't know whether or how this would also affect ipfw in-kernel nat.
>
> If in fact no ip6 packets are being passed to natd as rule 60 indicates,
> at least for traffic inbound to ipfw, then that is indeed strange, but I
> suspect that on the outbound pass, using this rather confusing 'skipto
> 500 .. keep-state' logic, perhaps some outbound ip6 packets may have
> been being passed to natd ..
>
> Could you check to see if it works only changing the _outbound_ divert
> rule from ip to ip4, leaving the inbound one at 'ip'?  If so, this would
> validate your original theory re rule 60, on the inbound pass.  If not,
> it may be a useful data point for resolving the problem.

ipfw add 99 divert natd ip from any to any in via $pif
ipfw delete 100

NATting still works

ipfw delete 500
ipfw add 500 divert natd ip  from any to any out via $pif

NATting broken

ipfw delete 500
ipfw add 500 divert natd ip  from any to any out via $pif

NATting works again

>
>  > > Meanwhile I found bug 148137 [ipfw] call order of natd and ipfw startup scripts
>  > > http://www.freebsd.org/cgi/query-pr.cgi?pr=148137&cat=conf
>  > > Don't know if that's directly related, but it may be worth a try to
>  > > revert back to the RELENG_8_0 script.
>  > >
>  > > Will let you now my findings.
>
> Did you try that, to see whether it was an issue?  More below ..

Sorry, didn't try reverting back to the 8.0 script. The ip -> ip4
change fixed it so I didn't need the other option.

>
>  > > Kind regards,
>  > >
>  > > Spil.
>  > >
>  > >
>  > > On Wed, Jul 21, 2010 at 2:57 PM, Sergey G Nasonov <snasonov at bcc.ru> wrote:
>  > >> Hello Spill,
>  > >>
>  > >> I have get the same trouble after updating my 8.0 Stable. I thing you need
>  > >> modify some firewall rules.
>  > >>
>  > >> Please change
>  > >>
>  > >> $cmd 100 divert natd ip from any to any in via $pif # Mangle inbound
>  > >>
>  > >> to
>  > >>
>  > >> $cmd 100 divert natd ip4 from any to any in via $pif # Mangle inbound
>  > >>
>  > >> and
>  > >>
>  > >> $cmd 500 divert natd ip from any to any out via $pif
>  > >>
>  > >> to
>  > >>
>  > >> $cmd 500 divert natd ip4 from any to any out via $pif
>  > >>
>  > >> accordingly.
>  > >>
>  > >> --
>  > >>
>  > >> Best Regards,
>  > >>
>  > >> Nasonov Sergey
>  > >
>  > >
>  > > On Wed, Jul 21, 2010 at 11:40 AM, Spil Oss <spil.oss at gmail.com> wrote:
>  > >> Hi,
>  > >>
>  > >> Testing FreeBSD 8.1 I noticed that I seem to have routing or nat or
>  > >> firewall issues. (csupped RELENG_8_1 which was -RELEASE not -RC last
>  > >> night?)
>  > >> - 8.1 booted fine
>  > >> - connections from the system itself were fine
>  > >> - connections from my jails to the internet were not working
>  > >> - connections from my LAN/WLAN to the internet were not working
>  > >> Reverting back to 8.0-p2 with the same configuration works fine.
>  > >>
>  > >> In UPDATING I see that rc.firewall and rc.firewall6 were unified.
>  > >>
>  > >> Setup is
>  > >> - xl0 connected to internet/public IP via dhcp
>  > >> - bge0/wlan0(ath0) connected to LAN
>  > >> - jails have ip's on bge0 in the same subnet as the LAN
>  > >> - allow all from any to any via bge0|wlan0|lo0
>
> The latter point looks problematic, see below.
>
>  > >> - NAT using natd
>  > >>
>  > >> My guess is that something's changed to ipfw that is affecting my
>  > >> network settings. Any clues where I went wrong?
>  > >>
>  > >> Help appreciated/ Kind regards,
>  > >>
>  > >> Spil.
>  > >>
>  > >> rc.conf:
>  > >> firewall_enable="YES"
>  > >> firewall_script="/etc/ipfw.rules"
>  > >>
>  > >> natd.conf
>  > >> interface xl0
>  > >> dynamic yes
>  > >> same_ports yes
>  > >> # http/https to http jail
>  > >> redirect_port tcp 192.168.2.3:80 80
>  > >> redirect_port tcp 192.168.2.3:443 443
>  > >>
>  > >> Part of /etc/ipfw.rules
>  > >> #!/bin/sh
>  > >> cmd="ipfw -q add"
>  > >> skip="skipto 500"
>  > >> pif=xl0
>  > >> pif6=gif0
>  > >> ext6="2001:dead:beef:1::1"
>  > >> ks="keep-state"
>  > >>
>  > >> ipfw -q -f flush
>  > >>
>  > >> # Allow internal traffic
>  > >> $cmd 002 allow all from any to any via bge0 # exclude LAN traffic
>  > >> $cmd 003 allow all from any to any via lo0  # exclude loopback traffic
>  > >> $cmd 004 allow all from any to any via wlan0 # exclude WLAN traffic
>  > >> $cmd 005 allow all from any to any via bridge0 # exclude WLAN traffic
>  > >> $cmd 006 allow all from any to any via tun0 # exclude WLAN traffic
>
> There are problems wih this, based on the apparent misunderstanding (by
> the present IPFW section's author) of how 'via' works when direction (in
> or out) and/or a specific interface (recv or xmit) is not specified.
>
> I'm assuming that apart from lo0, traffic coming in on some or all of
> bge0, wlan0, bridge0 and tun0 is to be translated by NAT if destined to
> be going out on the public interface xl0, right?  Certainly you've
> indicated that your jails are on bge0 with 192.168.x.x addresses, but
> these are local aliases which wouldn't show bge0 as the recv interface:

right! bge0, wlan0 and bridge0 should be translated by NAT.

I think I almost understand all below. Just can't figure out (yet) why
the jails have no problem accessing the internet. Many thanks for this
detailed explanation!  Will use it to my advantage.

>
>    "A packet may not have a receive or transmit interface: packets
>    originating from the local host have no receive interface, while
>    packets destined for the local host have no transmit interface."
>
> Let's consider just one initial packet coming in from bge0 from some
> private LAN address, other than your local jail IPs, that's destined for
> an outside address and so needs to be NAT'd before transmission.
>
> On the inbound pass, the packet comes in on bge0 and so satisfies 'via
> bge0' so is immediately allowed in, and that's it.  That may be ok, as
> NAT is here only to be performed on the outbound pass, but it could be
> anything, and there's no protection here against it being spoofed, as
> opposed to placement of NAT rules in the rc.firewall 'simple' ruleset.
>
> So having been allowed in, after kernel routing (but before NAT, as we
> haven't reached any check-state rule yet) it reenters the firewall for
> the outbound pass with its xmit interface set to xl0, but with its recv
> interface still set to bge0.
>
> Thus the unqualified 'via bge0' is still true, so this packet is passed
> immediately, with source address still set to the private LAN address,
> without having been translated by NAT.  As are similarly any packets
> bound for the outside initially coming in on wlan0, bridge0 or tun0 that
> originate from other than the local host.
>
> This could be obviated by replacing 'via $if' by 'in recv $if' in the
> rules above, so such packets going out pass through the firewall rules,
> including setting state for reply packets received on the outside iface.
>
> So as is, the rules below are only applied to outbound packets that are
> NOT received on those interfaces, which doesn't sound like what you'd
> intend, except packets generated by or destined for the local box.
>
>  > >> # Allow all encapulated IPv6 to/from tunnel PoP
>  > >> $cmd 010 allow ip4 from <tunnel-provider-ipv4> to me via $pif
>  > >> $cmd 010 allow ip4 from me to <tunnel-provider-ipv4> via $pif
>
> Taken on faith.
>
>  > >> # Black-hole some stuff using tables
>  > >> $cmd 050 drop ip from "table(17)" to any in via $pif
>  > >> $cmd 050 drop ip from any to "table(17)" out via $pif
>  > >>
>  > >> # Separate IPv6 rules (no NAT!)
>  > >> $cmd 060 skipto 1000 ip6 from any to any
>  > >>
>  > >> $cmd 100 divert natd ip from any to any in via $pif # Mangle inbound
>  > >> packets from external
>  > >> $cmd 101 check-state
>  > >>
>  > >> # Authorized outbound packets
>  > >> $cmd 130 $skip icmp from any to any out via $pif $ks
>  > >> $cmd 150 $skip tcp from any to any out via $pif $ks
>  > >> $cmd 151 $skip udp from any to any out via $pif $ks
>  > >>
>  > >> $cmd 200 allow udp from 10.50.0.1 to me 68 in $ks
>  > >>
>  > >> # Deny all inbound traffic from non-routable reserved address spaces
>  > >> $cmd 300 unreach host all from 192.168.0.0/16  to any in via $pif
>  > >> #RFC 1918 private IP
>  > >> $cmd 301 unreach host all from 172.16.0.0/12   to any in via $pif
>  > >> #RFC 1918 private IP
>  > >> $cmd 302 unreach host all from 10.0.0.0/8      to any in via $pif
>  > >> #RFC 1918 private IP
>  > >> $cmd 303 unreach host all from 127.0.0.0/8     to any in via $pif  #loopback
>  > >> $cmd 304 unreach host all from 0.0.0.0/8       to any in via $pif  #loopback
>  > >> $cmd 305 unreach host all from 169.254.0.0/16  to any in via $pif
>  > >> #DHCP auto-config
>  > >> $cmd 306 unreach host all from 192.0.2.0/24    to any in via $pif
>  > >> #reserved for docs
>  > >> $cmd 307 unreach host all from 204.152.64.0/23 to any in via $pif  #Sun cluster
>  > >> $cmd 308 unreach host all from 224.0.0.0/3     to any in via $pif
>  > >> #Class D & E multicast
>  > >>
>  > >> # Deny packets that did not match the dynamic rule table
>  > >> #$cmd 330 deny all from any to any frag in via $pif # All late fragments
>
> The author (here and in other writings) expounds that fragmented packets
> are necessarily bad, and indicate an attack of some sort.  Not in fact
> so; eg if you were using say zen.spamhaus.org as an RBL you'll be seeing
> valid fragmented UDP port 53 packets of around 2000 bytes total all day.
> I gather DNSSEC will also require acceptance of fragmented datagrams.
> The default rc.firewall rules pass IP frags, and I suggest you do too.
>
>  > >> #$cmd 332 deny tcp from any to any established in via $pif # Deny ACK
>  > >>
>  > >> # Authorized inbound packets
>  > >> $cmd 400 allow icmp from any to any icmptypes 0,11 # echo reply and TTL-exceeded
>
> Add icmptype 3 here to pass unreachables including path MTU discovery.
>
>  > >> $cmd 420 allow tcp from any to me ssh in via $pif setup $ks
>  > >> $cmd 421 allow tcp from any to me smtp in via $pif
>  > >> $cmd 422 allow tcp from any to me http in via $pif
>  > >> $cmd 423 allow tcp from any to me https in via $pif
>  > >> $cmd 424 allow tcp from any to me imaps in via $pif
>  > >>
>  > >> #$cmd 449 unreach host ip from any to any in via $pif
>  > >> $cmd 448 reject log all from any to any in via $pif
>  > >> $cmd 449 reject log all from any to any out via $pif
>  > >> $cmd 450 reject log ip from any to any
>
> Security-wise you'd be better off just dropping these using deny than
> reject (deprecated, equivalent to unreach host), here and above.
>
> However I don't wish to let such details obscure the more major issue.
>
>  > >>
>  > >> # This is skipto location for outbound stateful rules
>  > >> $cmd 500 divert natd ip from any to any out via $pif
>  > >> $cmd 510 allow ip from any to any
>
> cheers, Ian


More information about the freebsd-ipfw mailing list