Performance issues with vnet jails + epair + bridge

From: Sad Clouds <cryintothebluesky_at_gmail.com>
Date: Thu, 12 Sep 2024 17:16:18 UTC
Hi, I'm using FreeBSD-14.1 and on this particular system I only have a
single physical network interface, so I followed instructions for
networking vnet jails via epair and bridge, e.g.

devel
{
        vnet;
        vnet.interface = "e0b_devel";
        exec.prestart += "/jails/jib addm devel genet0";
        exec.poststop += "/jails/jib destroy devel";
}

The issue is bulk TCP performance throughput between this jail and the
host is quite poor, with one CPU spinning 100% in kernel and others
sitting mostly idle.

It seems there is some lock contention somewhere, but I'm not sure if
this is around vnet, epair or bridge subsystems. Are there
other alternatives for vnet jails? Can anyone recommend specific
deployment scenarios? I've seen references to netgraph which could be
used with jails. Does it have better performance and scalability and
could replace epair and bridge combination?

Thanks.