When is a switch not a switch?

[Jason Tubnor]

Hi,

On Tue, 20 Oct 2020 at 13:02, D'Arcy Cain <darcy at druid.net> wrote:

> I am using bhyve with vm-bhyve,  I am trying to set up a virtual network
> with multiple hosts.  The idea is that a VM would be on the same virtual
> network no matter which actual host it is on.
>
> Say I have a public network a.b.c.0/24.  I thought I could create a switch
> on a host.  The host would be a.b.c.1 and the VMs would be a.b.c.100 and
> a.b.c.101.  The idea would be that the VMs would appear on the real
> network.
>   Then the 101 VM could migrate to a.b.c.2 and still be accessible.  I
> envisioned some sort of proxy arp would happen so that every VM would
> simply
> announce itself wherever it was.
>

It looks like you are over complicating this.  When using vm-bhyve, as long
as each host has the same vswitch (bridge) then the tap will automagically
be inserted correctly on guest startup (as long as the conf file follows
your guest storage). Let vm-bhyve manage bridge creation.

Only use /etc/rc.conf to bring up the interface.  If you are running > 11.4
then you must turn LRO off (-lro) when you bring up the interface.  The
other settings in this thread can be left on.  The problem you are
experiencing is packet fragmentation that the guest has to deal with
because LRO is enabled (off by default in 11, enabled in 12 and above).
LRO should be disabled automatically when an interface (or child VLAN) is
added to a bridge.  I have tried to get the network guys to fix this but no
such luck.


> This did seem to work in that I could ping from the VM:
>
> # ping 8.8.8.8
> PING 8.8.8.8 (8.8.8.8): 56 data bytes
> 64 bytes from 8.8.8.8: icmp_seq=0 ttl=114 time=1.734 ms
>
> Even IPV6:
>
> # ping6 2605:2600:1001::4b
> PING6(56=40+8+8 bytes) 2605:2600:1001::4 --> 2605:2600:1001::4b
> 16 bytes from 2605:2600:1001::4b, icmp_seq=0 hlim=64 time=0.960 ms
> 16 bytes from 2605:2600:1001::4b, icmp_seq=1 hlim=64 time=0.415 ms
>
> However TCP doesn't work.  In fact, I could only ping by IP because the
> system couldn't connect to the DNS server, to get an address even though
> it
> could ping it.
>
> I guess my first question is does this seem doable?  If so, what am I
> missing?  Is it possible that a bhyve switch is more like a router?