Re: Review request: D60083 libalias: index fully specified inbound links by remote endpoint

hi!

I'm happy to land it if no-one has any objections.



-adrian


On Tue, 6 Oct 2026 at 07:28, Paulo Fragoso <paulo@nlink.com.br> wrote:

> Hi,
>
> I would like to ask for reviews of D60083:
>
>    https://reviews.freebsd.org/D60083
>
> Problem
>
> With ipfw nat (or ng_nat) and redirect_addr / redirect_port in front
> of a simple web service, every inbound connection to public:443 creates
> a link with alias_port = 443. All of them land in the same group_in,
> and _FindLinkIn() walks that group's grp->full list linearly for every
> packet.
>
> This hit a production hosting datacenter where I work as a
> consultant: one network thread was pinned at 100% with only a few
> hundred packets per second. We could not find the cause at the time.
> First we moved from ipfw to pf, and later the service was moved off
> FreeBSD to Debian/Proxmox. The datacenter lost a customer because of
> it. I found the root cause only later (two and half years later,
> with the application's source code and a lab that reproduces it.
>
> The behaviour is old: the same grouping existed in the pre-2021 hash
> table, as far back as libalias 1.0/2.1. Anyone running ipfw nat with
> redirects in front of a busy service can hit it.
>
> Change
>
> grp->full becomes an RB tree keyed by (dst_addr, dst_port), with a
> per-instance sequence number as a newest-first tie-break. That keeps
> the current semantics when several links share the same remote
> endpoint (3_natin:2_portoverlap depends on it). Partially specified
> links keep the existing list. The new field in struct libalias is
> appended at the end, because ipfw_nat.ko and ng_nat.ko read
> logDesc/packetAliasMode directly through alias_local.h.
>
> Results (16.0-CURRENT, VNET jails over epair, SYNs to one redirected
> port, 1000 pps):
>
>                            stock (~170k links)   patched (~141k links)
>    LibAliasIn per packet   1 to 2 ms             2 to 4 us
>    packets per 10 s        ~7,200 (saturated)    ~10,000 (all offered)
>    epair_task CPU          99%                   0.0 to 0.15%
>
> Userland, 100k links in one group: a new SYN goes from about 1.25 ms
> to about 600 ns it was really fast.
>
> I also have a backport to 14.0 tested in userland.
>
> adrian@ accepted it and asked for more networking people to look at
> it. I am happy to run more tests, for example ng_nat or UDP endpoint
> independent mapping, if anyone has a scenario in mind.
>
> Thanks,
> Paulo Fragoso
>
>