[Bug 229957] [epair] MAC addresses all the same, no randomness

[bugzilla-noreply at freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=229957

            Bug ID: 229957
           Summary: [epair] MAC addresses all the same, no randomness
           Product: Base System
           Version: CURRENT
          Hardware: Any
                OS: Any
            Status: New
          Severity: Affects Many People
          Priority: ---
         Component: kern
          Assignee: bugs at FreeBSD.org
          Reporter: ohartmann at walstatt.org

On recent CURRENT (FreeBSD 12.0-CURRENT #248 r336596: Sun Jul 22 09:31:53 CEST
2018 amd64), having vnet jails (VIMAGE kernel) owning their private epair
pseudo NIC and having their external end being part of a bridge(4), results in
a weird behaviour since CURRENT creates on ALL external parts of the epair pair
(b-part in my case) the very same MAC address! Each vnet jail then gets the
appropriate internal part of the epair pair (a-part in my case) but since I
group them on bridge(4) if-devices, then there is a MAC address problem and
this results in a very undpredictable, weirsd behaviour on FreeBSD (nothing is
reported to the console/log/kernel so far).

This is my ifconfig result after creating a bunch of epairs:


epair3b: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0
mtu 1500
        options=8<VLAN_MTU>
        ether 02:48:e2:b0:8c:0b
        groups: epair 
        media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
        status: active
epair52b: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0
mtu 1500
        options=8<VLAN_MTU>
        ether 02:48:e2:b0:8c:0b
        groups: epair 
        media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
        status: active
epair10013b: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric
0 mtu 1500
        options=8<VLAN_MTU>
        ether 02:48:e2:b0:8c:0b
        groups: epair 
        media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
        status: active
epair17b: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0
mtu 1500
        options=8<VLAN_MTU>
        ether 02:48:e2:b0:8c:0b
        groups: epair 
        media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
        status: active
epair10015b: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric
0 mtu 1500
        options=8<VLAN_MTU>
        ether 02:48:e2:b0:8c:0b
        groups: epair 
        media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
        status: active


One of the jails:

root at ns01:~ # ifconfig
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
        inet 127.0.0.1 netmask 0xff000000 
        groups: lo 
enc0: flags=0<> metric 0 mtu 1536
        groups: enc 
epair3a: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=8<VLAN_MTU>
        ether 02:48:e2:b0:8c:0a
        inet 192.168.0.3 netmask 0xffffff00 broadcast 192.168.0.255 
        groups: epair 
        media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
        status: active

And another jail on the very same bridge pseudo device:

root at db01:~ # ifconfig
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
        inet 127.0.0.1 netmask 0xff000000 
        groups: lo 
enc0: flags=0<> metric 0 mtu 1536
        groups: enc 
epair52a: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=8<VLAN_MTU>
        ether 02:48:e2:b0:8c:0a
        inet 192.168.0.52 netmask 0xffffff00 broadcast 192.168.0.255 
        groups: epair 
        media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
        status: active


There was an issue earlier with broken epair code on CURRENT (and prior) which
has been supposedly "fixed" a couple of time ago, but it seems the "fix" broke
more than it fixed.

I tried to solve the problem by applying manually MAC addresses via the
ifconfig "ether" option to guarantee different MACs on each collision doamain
but there is a weird behaviour now using IPFW. While the setup I use including
the workaround with the manually set "ether" on each epair works perfectly on
11.2-RELENG, I have trouble in CURRENT: Although OPEN firewall (ipfw) setting
in each jail, pinging the host owning the physical NIC which is part of the
bridge from any jail also member of the bridge doesn't work until the host
owning the physical NIC is pinging first that jail's address. I can not say
whether this is an IPFW issue or also related to the corrupt epair handling.

-- 
You are receiving this mail because:
You are the assignee for the bug.