Re: Forwarding packets to the host stack in net map lb app

From: Kim Shrier <kim_at_westryn.net>
Date: Mon, 26 Sep 2022 17:21:19 UTC
Thanks for your input.

In the interim, I have actually implemented your second (harder) option and it appears to be working quite well.  I am still testing but it does look promising.

Kim

> On Sep 26, 2022, at 10:05 AM, Vincenzo Maffione <vmaffione@freebsd.org> wrote:
> 
> I think you could avoid any modifications to lb(8) and take advantage of the "multiple pipe groups" feature.
> You open two groups, 
> # lb -i netmap:em0/R -p mon:$N -p fwd:$M [...]
> Each group receives all the packets arriving on the RX (NIC) rings of em0. (I'm pretty sure) this happens without packet copies, i.e. by swapping netmap slots.
> 
> The first group (mon) is for your existing monitor process. The second one (fwd) would be used for a separate process that handles the host stack:
>  - It reads from fwd:$M pipes, selecting only the RX packets that should be forwarded to the host stack. Selected packets will be forwarded to netmap:em0^/T. All the other packets are just dropped.
>  - it forwards all traffic from netmap:em0^/R to netmap:em0/T (e.g. from the em0 host RX ring to the em0 TX rings). Keep in mind that lb does not touch em0 TX rings, so there would not be conflicts. In any case, it is good practice to have lb only open RX rings (netmap:em0/R).
> This second process can probably be a modified version of the netmap bridge, although you have asymmetric three-party forwarding here (fwd/R --> netmap:em0^T, netmap:em0^/R --> netmap:em0/T).
> 
> The alternative (harder) option would be to actually modify lb(8). You should probably:
>  - open netmap:em0^/R and netmap:em0*/T with separate nmport_open() calls
>  - parse the packet before pkt_hdr_hash() to select the RX packets that you need to forward to the host TX ring, and modify the forwarding logic to perform this task.
>  - modify the logic of the lb poll() loop so that it also performs the forwarding from host RX rings to NIC TX rings
> I'm not sure that you would have any advantages by choosing this path.
> 
> Cheers,
>   Vincenzo
>