Re: Question about vm-bhyve

I forgot to mention another strange (to me) thing:
If (after the FreeBSD host is up) I stop the debian with:
"vm stop debian"
and then I restart the vm:
"vm start debian"
this time vm-bhyve *adds* the tap0 to the switch:

re0bridge: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
        options=0
        ether 58:9c:fc:00:32:71
        id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
        maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
        root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
        member: tap0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 5 priority 128 path cost 2000000
... ALL OTHER INTERFACES

I really do not understand why this is not done at system startup.
Maybe a reason of timing?
Maybe vm-bhyve tries to add the tap0 interface too early in the system startup sequence?
But I have tried to modify the /usr/local/etc/rc.d/vm startup script adding "bastille" to "# REQUIRE":

# REQUIRE: NETWORKING SERVERS dmesg bastille <= "bastille" added by me

but nothing changes.

Thank again.

Mauro.

> Il 13/07/2025 11:54 CEST trust_no_1@libero.it ha scritto:
> 
>  
> Hello everybody.
> I have a small pc with FreeBSD 14.3 and only 1 nic, re0.
> This pc has a number of jails (vnet) managed with bastille.
> Bastille creates the bridge (re0bridge) for the jails, and the all jails work well.
> 
> Now I would like to start a debian bhyve vm, managed with vm-bhyve.
> I have installed the vm, and the debian vm starts.
> I can login in it with:
> "vm console debian"
> but I can't manage to have the network working for the debian vm.
> I have read that vm-bhyve needs to be configured to use a "manual" switch to set up the network, so I created the switch with:
> "vm switch create -t manual -b re0bridge switch0vm"
> and the command creates the switch:
> 
> root@dukefleed:~ # vm switch list
> NAME       TYPE    IFACE      ADDRESS  PRIVATE  MTU  VLAN  PORTS
> switch0vm  manual  re0bridge  n/a      no       n/a  n/a   n/a
> 
> root@dukefleed:~ # vm switch info switch0vm
> ------------------------
> Virtual Switch: switch0vm
> ------------------------
>   type: manual
>   ident: re0bridge
>   vlan: -
>   physical-ports: -
>   bytes-in: 7928886 (7.561M)
>   bytes-out: 52934599 (50.482M)
> 
> Debian is configured to use this switch:
> 
> root@dukefleed:~ # cat /vm/debian/debian.conf
> loader="grub"
> cpu=1
> memory=2048M
> network0_type="virtio-net"
> network0_switch="switch0vm"
> disk0_type="ahci-hd"
> disk0_name="disk0.img"
> grub_run_partition="1"
> grub_run_dir="/boot/grub"
> uuid="0cd9f236-5808-11f0-bb94-94c6911cb3e4"
> network0_mac="58:9c:fc:0c:85:d0"
> 
> In rc.conf I have setup vm-bhyve to start debian:
> vm_enable="YES"
> vm_dir="zfs:zroot/vm"
> vm_list="debian"
> vm_delay="10"
> And indeed it is started, and the tap0 interface *is* created, but tap0 it is *not* added to the switch:
> 
> root@dukefleed:~ # vm list
> NAME    DATASTORE  LOADER  CPU  MEMORY  VNC  AUTO     STATE
> debian  default    grub    1    2048M   -    Yes [1]  Running (1477)
> 
> root@dukefleed:~ # ifconfig
> 
> re0: flags=1008943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
>         options=82099<RXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,LINKSTATE>
>         ether 94:c6:91:1c:b3:e4
>         inet 192.168.1.18 netmask 0xffff0000 broadcast 192.168.255.255
>         media: Ethernet autoselect (1000baseT <full-duplex>)
>         status: active
>         nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
> lo0: flags=1008049<UP,LOOPBACK,RUNNING,MULTICAST,LOWER_UP> metric 0 mtu 16384
>         options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
>         inet 127.0.0.1 netmask 0xff000000
>         inet6 ::1 prefixlen 128
>         inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2
>         groups: lo
>         nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
> bastille0: flags=8008<LOOPBACK,MULTICAST> metric 0 mtu 16384
>         options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
>         groups: lo
>         nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
> pflog0: flags=1000141<UP,RUNNING,PROMISC,LOWER_UP> metric 0 mtu 33152
>         options=0
>         groups: pflog
> re0bridge: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
>         options=0
>         ether 58:9c:fc:00:32:71
>         id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
>         maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
>         root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
>         member: e0a_bastille0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
>                 ifmaxaddr 0 port 11 priority 128 path cost 2000
> < OTHER INTERFACES REMOVED FOR CLEARITY >
>         groups: bridge
>         nd6 options=9<PERFORMNUD,IFDISABLED>
> e0a_bastille0: flags=1008943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
>         description: vnet host interface for Bastille jail dns
>         options=8<VLAN_MTU>
>         ether 02:20:98:1c:b3:e4
>         hwaddr 02:25:82:a8:15:0a
>         groups: epair
> < OTHER INTERFACES REMOVED FOR CLEARITY >
> tap0: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
>         description: vmnet/debian/0/re0bridge
>         options=4080000<LINKSTATE,MEXTPG>
>         ether 58:9c:fc:10:16:14
>         groups: tap vm-port
>         media: Ethernet 1000baseT <full-duplex>
>         status: active
>         nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
>         Opened by PID 1477
> 
> My question is: what I have missing?
> Maybe I have misunderstood how vm-bhyve works, and this is expected?
> Maybe it is not possible to have bastille+vm-bhyve both working on a pc with only 1 nic?
> Can you please enlight me?
> I have spent days google-ing and asking questions to many AI's, but now I have no more ideas.
> Thank you in advance and sorry for the english, I am italian.
> Greetings fron Italy.
> 
> Mauro.