epair/vbridge: no IPv6 traffic egress until first IPv6 packet flows in

From: FreeBSD User <freebsd_at_walstatt-de.de>
Date: Tue, 10 Oct 2023 17:26:22 UTC
Hello,

at first: observation is below, marked [OBSERVATION].

Running recent CURRENT (FreeBSD 15.0-CURRENT #26 main-n265831-3523f0677ef: Mon Oct  9 14:00:42
CEST 2023 amd64), I've configured a bridge (bridge0), the hosts's interface igb0 (I350-T2 two
port Gigabit Network Connection ) is member of that bridge and so a couple of epair(4) devices
belonging to a couple of jails.

IP filter is ipfw, each jail does have profile "WORKSTATION" configured and enabled, so the
host itself. On those hosts I use the standard rc facility.

Additionally, following MIB knobs are set to the following values:

net.link.bridge.pfil_local_phys: 0
net.link.bridge.pfil_member: 0
net.link.bridge.pfil_bridge: 0
net.link.bridge.pfil_onlyip: 0


and

net.link.ether.inet.allow_multicast: 0
net.link.ether.inet.log_arp_permanent_modify: 1
net.link.ether.inet.log_arp_movements: 1
net.link.ether.inet.log_arp_wrong_iface: 1
net.link.ether.inet.garp_rexmit_count: 0
net.link.ether.inet.max_log_per_second: 1
net.link.ether.inet.maxhold: 16
net.link.ether.inet.wait: 20
net.link.ether.inet.proxyall: 0
net.link.ether.inet.maxtries: 5
net.link.ether.inet.max_age: 1200
net.link.ether.arp.log_level: 6
net.link.ether.ipfw: 0

On epairs as well as on the main hosts igb0 NIC, both IPv4 and IPv6 are configured, IPv6 uses
ULA and setup doesn't has anything fancy.

Pinging and connecting to hosts "outside" the box of the host in question (all FreeBSD
CURRENT/14-STABLE) is possible without ANY(!) restriction or something nonregular.

[OBSERVATION]

JAILs can ping any(!) IPv4 host in the net, the main-jail-bearing-host (main-host) itself or
any host on the configured bridge() (important: jail -> main-host).

JAILs can ping IPv6 any host on the bridge() or in the net or in the internet, BUT NOT the
main-host (igb0-owner). NO IPv6 related service FROM jail TOWARD main-host is possible, it is
all blocked (and I do not know what blocks or how ...).

Disbaling IPFW via "ipfw disable firewall" on all jails and main-host doesn't have any effect
in this state.

BUT: if the main-host IPv6 pings a jail on that bridge (packet flowing into (the) jail/ingress
from jail's perspective), the jail itself suddenly is then able to ping or do network traffic
in IPv6! Other jails remain dead that way for IPv6 until I ping them. I haven't or I'm
unable to check(ed) why ipv6-icmp is mysteriously "opening" the connection. SSH'ing via
IPv6/TCPv6 from the main-host into a jail (tcp/22) works also perfectly and works as the
"openener" - a stuck ping -6 towards the main-host then immediately starts flowing.

I remember that such a similar problem occured somewhere around FreeBSD 10/11 and the problem
vanished somehow in the vain of development/patching.

Even disabling IPFW on all entities or switching to profile "open" doesn't help.

Some services, like LDAP, nslcd do not work from jails if the jail remains in the initial
state. That troubles me. Pinging via IPv6 icmp the jail seems to loosen all restraints, LDAP
works, ping works, everything on IPv6 is normal as expected. But the JAIL has to be pinged
from the outside first!

As stated, I'm out of ideas here and what troubles me most is the fact even LDAP doesn't work
until the jail is pinged via icmp6 first time.After this "init", everything is normal.

IPv4 seems to be unharmed ...

Thanks for your patience and reading,

Kind regards
oh

-- 
O. Hartmann