Large scale NAT with PF - some weird problem

[Ian FREISLICH]

Milan Obuch wrote:
> On Mon, 29 Jun 2015 08:58:38 +0200
> Daniel Hartmeier <daniel at benzedrine.ch> wrote:
> 
> > On Sun, Jun 28, 2015 at 10:06:09AM +0200, Milan Obuch wrote:
> > 
> > > So, now I am at 10.2-PRERELEASE, r284884, and the issue is still
> > > here. It is totally weird, just change of IP the device is being
> > > natted to makes the issue disappear for this particular customer,
> > > but as soon as this exact IP is used again, the issue is here again.
> > 
> > I'd go over the entire network config (pf.conf, pfctl -sa, rc.conf,
> > netstat -anr, ifconfig, arp -an) and look for any mistake, like a
> > typo or a netmask which isn't what you thought it is (like on an
> > alias), or for any weirdness related to that IP address.
> > 
> > Daniel
> 
> Thanks for hint, there is some logic in there, however
> 
> grep <apparently.offending.ip.address> /etc/*
> 
> yields nothing, it is never mentioned in any config, just as part of
> pool in pf.conf statement
> 
> nat on $if_ext from <net_int> to any -> $pool_ext round-robin sticky-address
> 
> It is not mentioned in 'pfctl -sa' output, 'arp -an' output,
> 'netstat -anr' output... nowhere.
> 
> I did not mention this box runs quagga for configuring network, mainly
> routing via OSPF, but I do not think it is relevant to the problem I
> see as this is basically userland process communicating with forwarding
> path in kernel to configure routing, nothing else, and, naturally, it
> does not work with this particular IP either. I should have seen it
> otherwise in some of above mentioned commands output, I think.
> 
> Just to repeat myself a bit, when this problematic state occurs, some
> intenal IP is translated to this one offending public IP, and
> communication is broken in such a way I see no returning packets from
> outside world on uplink interface in tcpdump even if I know they are
> there because I can ping some other box outside where I can verify that
> and they are there...
> 
> I just found some other strange, to me, thing - in 'pfctl -sa' output,
> section SOURCE TRACKING NODES, almost all entries are in form
> 
> <one internal IP net_int table> -> <one external IP from $pool_ext> ( states 
..., connections ..., rate ... )
> 
> but there are some of them with first IP being public, second one
> 0.0.0.0 - where they could come from? Also, there are only couple of
> them, but in one is something even a bit more weird - in parens is
> 'states 4294967295', which seems a bit absurd to me, also, worth to
> mention, it is 0xffffffff in hexadecimal, and this looks like some
> underflow issue in the code.

Try making your pool smaller.  With the number of NAT states you
mentioned earlier, you should easily manage with a /24.

Ian

-- 
Ian Freislich