Question that has dogged me for a while.
Rodney W. Grimes
freebsd-rwg at pdx.rh.CN85.dnsmgr.net
Thu May 4 19:44:41 UTC 2017
> On 5/4/2017 13:47, Rodney W. Grimes wrote:
> >> On 5/4/2017 12:12, Rodney W. Grimes wrote:
> >>>> Consider the following network configuration.
> >>>>
> >>>>
> >>>> Internet ------- Gateway/Firewall ---------- Inside network (including a
> >>>> web host)
> >>>> 70.16.10.1/28 192.168.0.0/24
...
> > It is almost impossible to remotly debug this type of stuff without a
> > complete and full picture of all elements involved.
> > As a minimum:
> > ifconfig -a
> > ipfw -a list
> > sysctl net.inet.ip.fw.one_pass
> > sysctl net.inet.ip.forwarding
> >
> > I know this can be made to work, I think even dd-wrt has it right....
> > And here is a good jumping off point from a very quick google:
> > http://www.nycnetworkers.com/real-world/nat-reflectionnat-loopbacknat-hairpinning/
> root at IPGw:/usr/local/etc # ifconfig -a
> lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
> options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
> inet6 ::1 prefixlen 128
> inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1
> inet 127.0.0.1 netmask 0xff000000
> groups: lo
> nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
> ue0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
> options=80009<RXCSUM,VLAN_MTU,LINKSTATE>
> ether b8:27:eb:4e:88:64
> inet 192.168.10.200 netmask 0xffffff00 broadcast 192.168.10.255
> media: Ethernet autoselect (100baseTX <full-duplex>)
> status: active
> nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
> ue1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
> options=8000b<RXCSUM,TXCSUM,VLAN_MTU,LINKSTATE>
> ether 00:50:b6:5d:1d:9f
> inet 70.169.168.7 netmask 0xffffff80 broadcast 70.169.168.127
> media: Ethernet autoselect (100baseTX <full-duplex>)
> status: active
> nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
> ue0.3: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
> ether b8:27:eb:4e:88:64
> inet 192.168.4.200 netmask 0xffffff00 broadcast 192.168.4.255
> groups: vlan
> vlan: 3 vlanpcp: 0 parent interface: ue0
> media: Ethernet autoselect (100baseTX <full-duplex>)
> status: active
> nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
>
> root at IPGw:/usr/local/etc # ipfw -a list
> 00100 14 1042 allow ip from any to any via lo0
> 00200 0 0 deny log ip from any to 127.0.0.0/8
> 00300 0 0 deny log ip from 127.0.0.0/8 to any
> 00400 0 0 deny log ip from any to ::1
> 00500 0 0 deny log ip from ::1 to any
> 02000 0 0 allow ip from 192.168.100.1 to any in via ue1
> 02010 0 0 deny log ip from 192.168.0.0/16 to any not ipsec in
> via ue1
> 02020 0 0 deny log ip from 70.169.168.0/25 to any in via ue0
> 03000 0 0 deny log ip from 70.169.168.0/25 to any recv ue0
> 04000 0 0 deny log ip from table(22) to any recv ue1
> 04010 0 0 deny ip from any to
> 114.215.179.104,122.226.84.253,122.248.234.207,167.206.87.147,168.1.83.89,175.41.238.100,176.58.116.160,202.96.134.133,203.143.89.106,220.181.111.147,23.234.53.61,23.234.53.67,46.137.188.54,50.19.254.134,50.7.114.59,50.7.124.48,50.7.176.18,50.7.235.90,50.7.44.82,61.188.37.216,68.192.249.119,74.125.31.99
> 04020 0 0 deny log ip from
> 218.90.0.0/16,218.91.0.0/16,218.92.0.0/16,218.93.0.0/16,218.94.0.0/16 to
> any via ue1
> 05000 0 0 deny log ip from
> 10.0.0.0/8,172.16.0.0/12,192.168.0.0/16 to any not ipsec recv ue1
> 05010 0 0 deny log ip from
> 0.0.0.0/8,169.254.0.0/16,192.0.2.0/24,224.0.0.0/4,240.0.0.0/4 to any
> recv ue1
> 06000 8726 10333291 nat 100 ip4 from any to me recv ue1
> 07000 0 0 check-state :default
> 08000 21 1064 nat 200 ip4 from 192.168.0.0/16 to 70.169.168.7
Where is the other half of nat 200? This is from inside to outside IP,
there needs to be a return nat occuring to de Nat the packets
ipfw add 8000 nat 200 ip4 from 192.168.0.0/16 to 192.168.10.200,192.168.4.200
It takes 2 rules to the same NAT to have working NAT usually, one for
outbound packets, and one for inbound packets (relative to the NAT instance).
Do we see atleast the packets this nats on the wire with tcpdump?
> 08001 4834 264258 nat 100 ip4 from 192.168.0.0/16 to any xmit ue1
> 08009 0 0 deny log ip4 from 192.168.0.0/16 to any xmit ue1
> 08010 4836 264410 allow ip4 from 70.169.168.0/25 to any xmit ue1
> 08011 0 0 allow log ip from 192.168.10.200 to 192.168.0.0/16
> dst-port 2552
> 08020 5374 306553 allow ip from 192.168.0.0/16 to any recv ue0
> 08030 2 104 allow ip from 192.168.4.0/25 to any recv ue0.3
> 08500 0 0 deny log ip from 192.168.0.0/16 to any xmit ue1
> 09000 17823 20712366 allow ip from any to 192.168.0.0/16
> 22000 0 0 allow tcp from any to any established
Interesting that the count on this is 0? This is usually a stateless
packet matching rule that goes with your setups. Nvm, there are not
packets maching the setup rules, so no change to have this matter.
> 22700 0 0 allow tcp from any to me dst-port 2200 setup
> 22710 0 0 allow tcp from any to me dst-port 22 setup
> 22800 0 0 allow icmp from any to me
> 23100 0 0 allow udp from any to me dst-port 33434-34000
> 23110 0 0 allow udp from any 33434-34000 to me
> 23410 0 0 allow udp from any to me dst-port 53
> 23420 0 0 allow udp from me 53 to any
> 23430 4 545 allow udp from any 53 to me
> 23500 0 0 allow tcp from any to 192.168.1.214 dst-port 8080 setup
> 23510 0 0 allow tcp from any to 192.168.4.210 dst-port 443 setup
> 23520 0 0 allow tcp from any to 192.168.4.211 dst-port 443 setup
> 23530 0 0 allow tcp from any to 192.168.4.211 dst-port 554 setup
> 24430 0 0 allow udp from any 123 to me dst-port 123
> 24500 0 0 allow udp from any to me dst-port 500
> 24510 0 0 allow udp from me 500 to any
> 24520 0 0 allow udp from any to me dst-port 4500
> 24530 0 0 allow udp from me 4500 to any
> 24600 46 2760 deny tcp from 192.168.4.211 to any dst-port 80 setup
What are these denied packets? Part of our issue?
> 29999 5 272 deny log ip from any to any
And these?
> 65535 2603 379766 deny ip from any to any
>
> onepass is 0, forwarding is 1 of course.
> root at IPGw:/usr/local/etc # sysctl -a|grep forwarding
> net.inet.ip.forwarding: 1
> net.inet6.ip6.forwarding: 0
> root at IPGw:/usr/local/etc # sysctl -a | grep one_pass
> net.inet.ip.fw.one_pass: 0
>
> If it matters this is running on -HEAD (it's running on a PI3, so -HEAD
> is a mandate at this point.) The 0.3 interface is a VLAN for things
> that I have DMZd off so they can't play "send back data to poppa and
> scan the LAN" games (think consumer appliances.)
>
> Line 8000 does have packets that match. There IS a "check-state" right
> above it, but that shouldn't kill the output side -- and I moved it to
> 10000 (below the pass lines) without effect, just in case it was. NAT
> is working perfectly well for someone on the internal network but
> connecting to something outside on the Internet and the "hole punches"
> for a connection outside to the interior host work as well. Note that
> the line 8011, which SHOULD trap a "telnet 70.169.168.7 2552" from the
> inside network (and which DOES generate the packet counts on line 8000)
> does NOT register counts nor log anything, so whatever is nailing it
> it's happening before it gets there -- which is why I'm confused here.
>
> --
> Karl Denninger
> karl at denninger.net <mailto:karl at denninger.net>
> /The Market Ticker/
> /[S/MIME encrypted email preferred]/
--
Rod Grimes rgrimes at freebsd.org
More information about the freebsd-ipfw
mailing list