Large scale NAT with PF - some weird problem

[Ermal Luçi]

On Tue, Jun 23, 2015 at 10:12 AM, Milan Obuch <freebsd-pf at dino.sk> wrote:

> On Tue, 23 Jun 2015 09:49:57 +0200
> Ian FREISLICH <ian.freislich at capeaugusta.com> wrote:
>
> > Milan Obuch wrote:
> > > As a first step, I did small upgrade, so now I run FreeBSD
> > > 9.3-STABLE #0 r284695: Mon Jun 22 08:55:29 CEST 2015.
> > >
> > > I still see the issue, but I found simpler workaround when bad state
> > > ocurs - using
> > >
> > > pfctl -k <ip.of.affected.client>
> > > pfctl -K <ip.of.affected.client>
> > >
> > > in this order seems to remedy the issue for this one affected client
> > > without affecting other clients. This still does not solve the
> > > problem, just eases the reaction.
> >
> > How is your NAT rule defined?  I had a closer look at the way I did
> > it:
> >
> > nat on vlan46 from 10.8.0.0/15 to !<on-our-net> -> xx.xx.xx.xx/24
> > round-robin sticky-address
> >
> > I think you may be missing the "round-robin" that spreads the mapping
> > over your pool.  The manual says that when more than 1 address is
> > specified, round-robin is the only pool type allowed, it does not
> > say that when more than 1 address is specified this is the default
> > pool option.
> >
>
> Thanks for hint, however, this is not the case I think. My definition is
>
> nat on $if_ext from <net_int> to any -> $pool_ext round-robin
> sticky-address
>
> where <net_int> contains contains some /24 segments from 10.0.0.0/8
> range and one /24 and one /15 segment from 172.16.0.0/12 range,
> $pool_ext is one /23 public segment.
>
> > You can check your state table to see if it is indeed round-robin.
> >
> > #pfctl -s sta |grep " ("
> > ...
> > all tcp a.b.c.d:53802 (10.0.0.220:42808) -> 41.246.55.66:24
> > ESTABLISHED:ESTABLISHED all tcp a.b.c.e:60794 (10.0.0.38:47825) ->
> > 216.58.223.10:443       ESTABLISHED:FIN_WAIT_2
> >
> > If all your addresses "a.b.c.X" are the same, it's not round-robin
> > and that's your problem.
> >
>
> Well, this is something I do not fully understand. If my pool were
> a.b.c.0/24, then what you wrote could not be any other way - I think
> this is not what you meant. Or did you think there will be only one IP
> used? That's definitelly not the case, I see many IPs from my /23
> segment here.
>
> One strange thing occured, however - it looks like if one IP from
> this /23 range gets used, trouble occurs. I do pfctl -k and pfctl -K
> for this address and all is well again. As long as this one IP is not
> used, everything works. When it gets used again, voila, trouble again.
>
>
Can you check if you are reaching the limits on source entries
       set limit src-nodes 2000

           sets the maximum number of entries in the memory pool used for
           tracking source IP addresses (generated by the sticky-address and
           src.track options) to 2000.


> As this does not occur that fast, I need to check every now and then,
> and I am checking the other way too, but it is really annoying if it
> hits any customer.
>
> Regards,
> Milan
> _______________________________________________
> freebsd-pf at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-pf
> To unsubscribe, send any mail to "freebsd-pf-unsubscribe at freebsd.org"
>
> --
> Ermal
>