Large scale NAT with PF - some weird problem

[Milan Obuch]

On Mon, 29 Jun 2015 09:49:10 +0200
Ian FREISLICH <ian.freislich at capeaugusta.com> wrote:

> Milan Obuch wrote:
> > On Mon, 29 Jun 2015 08:58:38 +0200
> > Daniel Hartmeier <daniel at benzedrine.ch> wrote:
> > 
> > > On Sun, Jun 28, 2015 at 10:06:09AM +0200, Milan Obuch wrote:
> > > 
> > > > So, now I am at 10.2-PRERELEASE, r284884, and the issue is still
> > > > here. It is totally weird, just change of IP the device is being
> > > > natted to makes the issue disappear for this particular
> > > > customer, but as soon as this exact IP is used again, the issue
> > > > is here again.
> > > 
> > > I'd go over the entire network config (pf.conf, pfctl -sa,
> > > rc.conf, netstat -anr, ifconfig, arp -an) and look for any
> > > mistake, like a typo or a netmask which isn't what you thought it
> > > is (like on an alias), or for any weirdness related to that IP
> > > address.
> > > 
> > > Daniel
> > 
> > Thanks for hint, there is some logic in there, however
> > 
> > grep <apparently.offending.ip.address> /etc/*
> > 
> > yields nothing, it is never mentioned in any config, just as part of
> > pool in pf.conf statement
> > 
> > nat on $if_ext from <net_int> to any -> $pool_ext round-robin
> > sticky-address
> > 
> > It is not mentioned in 'pfctl -sa' output, 'arp -an' output,
> > 'netstat -anr' output... nowhere.
> > 
> > I did not mention this box runs quagga for configuring network,
> > mainly routing via OSPF, but I do not think it is relevant to the
> > problem I see as this is basically userland process communicating
> > with forwarding path in kernel to configure routing, nothing else,
> > and, naturally, it does not work with this particular IP either. I
> > should have seen it otherwise in some of above mentioned commands
> > output, I think.
> > 
> > Just to repeat myself a bit, when this problematic state occurs,
> > some intenal IP is translated to this one offending public IP, and
> > communication is broken in such a way I see no returning packets
> > from outside world on uplink interface in tcpdump even if I know
> > they are there because I can ping some other box outside where I
> > can verify that and they are there...
> > 
> > I just found some other strange, to me, thing - in 'pfctl -sa'
> > output, section SOURCE TRACKING NODES, almost all entries are in
> > form
> > 
> > <one internal IP net_int table> -> <one external IP from $pool_ext>
> > ( states 
> ..., connections ..., rate ... )
> > 
> > but there are some of them with first IP being public, second one
> > 0.0.0.0 - where they could come from? Also, there are only couple of
> > them, but in one is something even a bit more weird - in parens is
> > 'states 4294967295', which seems a bit absurd to me, also, worth to
> > mention, it is 0xffffffff in hexadecimal, and this looks like some
> > underflow issue in the code.
> 
> Try making your pool smaller.  With the number of NAT states you
> mentioned earlier, you should easily manage with a /24.
> 

OK, I changed pool_ext size from /23 to /24... just would like to know,
why should this have desired effect, please...

So I am going to observe how it works, but I am sure I had this pool
defined for some maybe years and did not receive any complaint until
just recently.

Regards,
Milan