Re: issue while bridging VF on mellanox
- Reply: benoitc@enki-multimedia.eu: "Re: issue while bridging VF on mellanox"
- Go to: [ bottom of page ] [ top of archives ] [ this month ]
Date: Mon, 17 Mar 2025 16:08:47 UTC
Hi Benoit, one more thing, Im not sure whats the current support for v6, specially for ND. Best regards. Santi On 3/17/25 16:48, Santiago Martinez wrote: > > Hi Benoit, > > I have a similar setup to yours (not using MLX4), and it works. > > I have two VMs acting as routers (R-A and R-B), each with two interfaces. > > On Router-A: > > * VTNET1 → P2P to Router-B. > * VTNET0 → Sub-interface (VLAN 333) - Client-A. > * VXLAN0 → Towards Router-B. > * BR-TEST → VXLAN0 and VTNET0.333 as members. > > On Router-B: > > * VTNET1 → P2P to Router-A. > * VTNET0 → Untagged - Client-B. > * VXLAN0 → Towards Router-A. > * BR-TEST → VXLAN0 and VTNET0 as members. > > Traffic passes. > > One thing to note—VXLAN0 will appear as UP/Active even if it’s not > reaching the other side. > > Best regards, > Santi > > > On 3/17/25 15:32, Benoit Chesneau wrote: >> Hi all, >> >> I'm trying to make some kind of cross connect between a vlan and a vxlan, but as far as I understand the mac address are not learnt from the vlan When I listen using tcpdump on the vlan i can see the arp request but not in the in the vxlan. ANy idea how should I fix this? Is there any other way to do a cross conect or shoud I go back to not use sriov when i need to build bridges? >> >> >> Any idea is welcome. >> >> Benoit >> >> More details follow >> >> The vlan is created over a VF device in a bhyve vm: >> >> vlan601: flags=1008943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1356 >> options=18080000<LINKSTATE,TXTLS4,TXTLS6> >> ether 4a:75:f1:f5:a5:0a >> inet6 fe80::4875:f1ff:fef5:a50a%vlan601 prefixlen 64 scopeid 0x5 >> groups: vlan >> vlan: 601 vlanproto: 802.1q vlanpcp: 0 parent interface: mce0 >> media: Ethernet 25GBase-SR <full-duplex,rxpause,txpause> >> status: active >> nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL> >> >> The vxlan is >> >> vxlan0: flags=1008903<UP,BROADCAST,PROMISC,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1356 >> options=80020<JUMBO_MTU,LINKSTATE> >> ether 58:9c:fc:10:12:01 >> inet6 fe80::5a9c:fcff:fe10:1201%vxlan0 prefixlen 64 tentative scopeid 0x7 >> groups: vxlan >> vxlan vni 400 local [2001:db8:1000::1]:4789 remote [2001:db8:1000::2]:4789 >> media: Ethernet autoselect (autoselect <full-duplex>) >> status: active >> nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL> >> >> >> And the bridge: >> >> # ifconfig bridge0 >> bridge0: flags=1028943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST,PPROMISC,LOWER_UP> metric 0 mtu 1356 >> options=0 >> ether 58:9c:fc:10:ff:d3 >> id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15 >> maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200 >> root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0 >> member: vlan601 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP> >> ifmaxaddr 0 port 5 priority 128 path cost 800 >> member: vxlan0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP> >> ifmaxaddr 0 port 7 priority 128 path cost 55 >> groups: bridge >> nd6 options=3<PERFORMNUD,ACCEPT_RTADV> >> >> The PF on the machine is a Mellanox [ConnectX-4 Lx >> >> # pciconf -lvbce mlx5_core0 >> mlx5_core0@pci0:17:0:0: class=0x020000 rev=0x00 hdr=0x00 vendor=0x15b3 device=0x1015 subvendor=0x1590 subdevice=0x00d4 >> vendor = 'Mellanox Technologies' >> device = 'MT27710 Family [ConnectX-4 Lx]' >> class = network >> subclass = ethernet >> bar [10] = type Prefetchable Memory, range 64, base 0xe4000000, size 33554432, enabled >> cap 10[60] = PCI-Express 2 endpoint max data 256(512) FLR RO NS >> max read 4096 >> link x8(x8) speed 8.0(8.0) ASPM L1(L1) >> cap 03[48] = VPD >> cap 11[9c] = MSI-X supports 64 messages, enabled >> Table in map 0x10[0x2000], PBA in map 0x10[0x3000] >> cap 09[c0] = vendor (length 24) >> cap 01[40] = powerspec 3 supports D0 D3 current D0 >> ecap 0001[100] = AER 1 0 fatal 0 non-fatal 1 corrected >> ecap 000e[150] = ARI 1 >> ecap 0010[180] = SR-IOV 1 IOV enabled, Memory Space enabled, ARI enabled >> 8 VFs configured out of 16 supported >> First VF RID Offset 0x0002, VF RID Stride 0x0001 >> VF Device ID 0x1016 >> Page Sizes: 4096 (enabled), 8192, 16384, 32768, 65536, 131072, 262144, 524288, 1048576, 2097152, 4194304 >> iov bar [1a4] = type Prefetchable Memory, range 64, base 0xc7ffe000000, size 1048576, enabled >> ecap 0019[1c0] = PCIe Sec 1 lane errors 0 >> ecap 000d[230] = ACS 1 Source Validation unavailable, Translation Blocking unavailable >> P2P Req Redirect unavailable, P2P Cmpl Redirect unavailable >> P2P Upstream Forwarding unavailable, P2P Egress Control unavailable >> P2P Direct Translated unavailable, Enhanced Capability unavailable >> PCI-e errors = Correctable Error Detected >> Unsupported Request Detected >> Corrected = Advisory Non-Fatal Error >> >> Other side is the same except that the card is an intel one: >> >> # pciconf -lvbce ixl0 >> ixl0@pci0:23:0:0: class=0x020000 rev=0x02 hdr=0x00 vendor=0x8086 device=0x1572 subvendor=0x15d9 subdevice=0x093b >> vendor = 'Intel Corporation' >> device = 'Ethernet Controller X710 for 10GbE SFP+' >> class = network >> subclass = ethernet >> bar [10] = type Prefetchable Memory, range 64, base 0xc4800000, size 8388608, enabled >> bar [1c] = type Prefetchable Memory, range 64, base 0xc5808000, size 32768, enabled >> cap 01[40] = powerspec 3 supports D0 D3 current D0 >> cap 05[50] = MSI supports 1 message, 64 bit, vector masks >> cap 11[70] = MSI-X supports 129 messages, enabled >> Table in map 0x1c[0x0], PBA in map 0x1c[0x1000] >> cap 10[a0] = PCI-Express 2 endpoint max data 256(2048) FLR RO >> max read 512 >> link x8(x8) speed 8.0(8.0) >> cap 03[e0] = VPD >> ecap 0001[100] = AER 2 0 fatal 0 non-fatal 1 corrected >> ecap 0003[140] = Serial 1 b8f678ffff55c27c >> ecap 000e[150] = ARI 1 >> ecap 0010[160] = SR-IOV 1 IOV enabled, Memory Space enabled, ARI enabled >> 16 VFs configured out of 64 supported >> First VF RID Offset 0x0010, VF RID Stride 0x0001 >> VF Device ID 0x154c >> Page Sizes: 4096 (enabled), 8192, 65536, 262144, 1048576, 4194304 >> iov bar [184] = type Prefetchable Memory, range 64, base 0xc5000000, size 65536, enabled >> iov bar [190] = type Prefetchable Memory, range 64, base 0xc5100000, size 16384, enabled >> ecap 0017[1a0] = TPH Requester 1 >> ecap 000d[1b0] = ACS 1 Source Validation unavailable, Translation Blocking unavailable >> P2P Req Redirect unavailable, P2P Cmpl Redirect unavailable >> P2P Upstream Forwarding unavailable, P2P Egress Control unavailable >> P2P Direct Translated unavailable, Enhanced Capability unavailable >> ecap 0019[1d0] = PCIe Sec 1 lane errors 0 >> PCI-e errors = Correctable Error Detected >> Unsupported Request Detected >> Corrected = Advisory Non-Fatal Error