When is a switch not a switch?

[Patrick M. Hausen]

Hi all,

> Am 20.10.2020 um 12:50 schrieb D'Arcy Cain <darcy at druid.net>:
> 
> On 10/20/20 5:36 AM, Patrick M. Hausen wrote:
>>> I did see that.  Does that mean that I don't even need to create switches at all?
>> What is a switch in this context? I use bridge interfaces to connect jails via epair
>> and VMs via tap.
> 
> When I started I thought of a switch as analogous to a physical switch.  If I am in an office with one ethernet jack but I have multiple devices I might connect a switch (or hub) to the jack and plug my devices into the switch. I don't need to create a separate network for my office.  All of my devices are on the company network.

OK, the "switch" interface in FreeBSD is bridge(4).

Or to cite Radia Perlman:

A bridge is a network device making forwarding decisions based on layer 2 addresses.
A router is a network device making forwarding decisions based on layer 3 addresses.
"Switch" is a marketing term meaning "faster or cheaper than the competition".

> cloned_interfaces="bridge0"
> ifconfig_bridge0="a.b.c.d.1 addm bge0 addm switch0 up"

> Except that switch0 doesn't get created until vm-bhyve starts so it probably doesn't exist at that time.

What is "switch0"? I suspect it is just a bridge interface that gets renamed by
your VM management software. In that case manually creating bridge0
and all the things we discussed will not get you anywhere.

>> If em0 does not have an IP address on the host and should be used
>> exclusively for VMs, then the bridge does not need an IP address, either.
>> Still you need to configure em0 "up".
> 
> I can't imagine a scenario like that.  You probably always need access to the host for maintenance.

Well, there could be a second hardware interface for host communication ...
And if one of the two is member of the bridge and the other one isn't it is
perfectly valid to plug them into the same broadcast domain and get
e.g. 1Gbit/s for the host and 1Gbit/s for all the jails or VMs.

>> And additionally ...
>> - you should disable all hardware acceleration features on the physical interface
> 
> Like ASF?

Real life example from our environment:

ifconfig_igb0="-rxcsum -rxcsum6 -txcsum -txcsum6 -tso -vlanhwtag -vlanhwtso up"
cloned_interfaces="bridge0"
ifconfig_bridge0_name="inet0"
ifconfig_inet0="addm igb0 up"
ifconfig_inet0_alias0="inet 1.2.3.4/24"

Then we configure iocage to attach the jails to bridge0.

In your case you would have to tell your VM management tool to attach the
VM tap interfaces to bridge0 instead of creating its own "switch0" - which I
suspect is a bridge interface in disguise. As you can see above we rename
all our Internet facing interfaces to "inet0" on all hosts. Then there are more
like "mgmt0", "priv0", ... like that. So probably the bridge is renamed to "switch0".

Tell the tool not to do that and use the preconfigured bridge0 instead.

Another useful sysctl to get reproduceable static MAC addresses for the bridge
itself accross reboots is:

loader.conf: if_bridge_load="YES"
sysctl.conf: net.link.bridge.inherit_mac=1


HTH,
Patrick
--
punkt.de GmbH
Patrick M. Hausen
.infrastructure

Kaiserallee 13a
76133 Karlsruhe

Tel. +49 721 9109500

https://infrastructure.punkt.de
info at punkt.de

AG Mannheim 108285
Geschäftsführer: Jürgen Egeling, Daniel Lienert, Fabian Stein
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: Message signed with OpenPGP
URL: <http://lists.freebsd.org/pipermail/freebsd-virtualization/attachments/20201020/07028379/attachment.sig>