bhyve windows guest unable to ping default gateway or external networks

Tue Nov 15 13:04:47 UTC 2016

Hello, and thanks for the suggestion.

I ran the following:
sudo ifconfig em0 -rxcsum -txcsum -vlanmtu -vlanhwcsum -vlanhwtso -tso4

which resulted in ifconfig showing:
em0: flags=8843 metric 0 mtu 1500
 options=2098

It looks like it removed rxcsum, txcsum, vlanhwtso and tso, but vlanhwcsum and vlan_mtu were not removed.

At this pointed I started up the VM and performed the same tests, but unfortunately the results were still the same.

I then directly applied your config to my ifconfig_em0 line, resulting in this rc.conf entry:
ifconfig_em0="inet 46.105.xxx.xxx netmask 0xffffff00 -rxcsum -txcsum -vlanmtu -vlanhwtso -tso4 -tso6 -vlanhwtso -vlanhwcsum"

Upon rebooting my ifconfig output was still the same as before the reboot. Just to be sure it didn't fix it, I booted up the VM again and tried to ping the default gateway, and it failed.

November 15, 2016 12:07 PM, "Rajil Saraswat"  wrote:
	On Nov 15, 2016 5:58 AM,  wrote:
>
> Hello,
>
> I am trying to get a windows server VM running under bhyve. So far I can use VNC to install Windows, but I've been unable to get the networking working. I will provide details below, and include a summary up here. On the guest, I have installed the latest version of winvirtio, and I have configured static IP addressing. The guest can ping itself, but not the host machine or anything external. It is worth noting that I am also currently running a CentOS guest, and this can connect to both the host and the outside world with no issues.
>
> Host: FreeBSD 11.0-RELEASE
> Guest: Tried both Windows Server 2016 Essentials and Windows Server 2012 Standard
>
> bhyve command:
> bhyve -c 1 -H -w -m 3G -s 0,hostbridge -s 3,ahci-cd,/iohyve/ISO// -s '4,ahci-hd,/dev/zvol/zroot/iohyve/win12/disk0,sectorsize=512' -s 10,virtio-net,tap2 -s '11,fbuf,tcp=0.0.0.0:5902 (http://0.0.0.0:5902),w=800,h=600,wait' -s 12,xhci,tablet -s 31,lpc -l com1,/dev/nmdm3A -l bootrom,/iohyve/Firmware/BHYVE_UEFI.fd/BHYVE_UEFI.fd ioh-win12
>
> ifconfig output:
> em0: flags=8843 metric 0 mtu 1500
> options=4219b
> ether e0:69:95:a2:f1:d9
> inet 46.105.xxx.xxx netmask 0xffffff00 broadcast 46.105.124.255
> inet6 fe80::e269:95ff:xxxx:f1d9%em0 prefixlen 64 scopeid 0x1
> inet6 2001:41d0:2:xxxx::1 prefixlen 56
> nd6 options=21
> media: Ethernet autoselect (100baseTX )
> status: active
> lo0: flags=8049 metric 0 mtu 16384
> options=600003
> inet6 ::1 prefixlen 128
> inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2
> inet 127.0.0.1 netmask 0xff000000
> nd6 options=21
> groups: lo
> bridge0: flags=8843 metric 0 mtu 1500
> ether 02:cf:0c:8f:13:00
> inet6 fd00::ffff prefixlen 112
> inet 10.0.0.254 netmask 0xffffff00 broadcast 10.0.0.255
> nd6 options=1
> groups: bridge
> id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
> maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
> root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
> member: tap2 flags=143
> ifmaxaddr 0 port 7 priority 128 path cost 2000000
> member: tap1 flags=143
> ifmaxaddr 0 port 5 priority 128 path cost 2000000
> member: tap0 flags=143
> ifmaxaddr 0 port 4 priority 128 path cost 2000000
> pflog0: flags=0 metric 0 mtu 33160
> groups: pflog
> tap0: flags=8943 metric 0 mtu 1500
> description: iohyve-centos
> options=80000
> ether 00:bd:5d:d3:e8:00
> nd6 options=21
> media: Ethernet autoselect
> status: active
> groups: tap
> Opened by PID 96206
> tap1: flags=8902 metric 0 mtu 1500
> description: iohyve-win16es
> options=80000
> ether 00:bd:61:ab:eb:01
> nd6 options=21
> media: Ethernet autoselect
> status: no carrier
> groups: tap
> tap2: flags=8943 metric 0 mtu 1500
> description: iohyve-win12
> options=80000
> ether 00:bd:c9:1a:59:02
> nd6 options=21
> media: Ethernet autoselect
> status: active
> groups: tap
> Opened by PID 1911
>
> It is using NAT with PF to reach the internet. It should not rely on NAT to be able to ping the host machine, but I will post the NAT configuration below anyway:
>
> /etc/pf.conf:
> # Automatically NAT any VMs
> nat on em0 inet from bridge0:network to any -> (em0)
>
>
> (On the Guest)
> ==============
>
> C:UsersAdministrator>ipconfig
>
> Ethernet adapter Ethernet:
>
> IPv4 Address: 10.0.0.3
> Subnet Mask: 255.255.255.0
> Default Gateway: 10.0.0.254
>
> C:UsersAdministrator>ping 10.0.0.3
>
> Pinging 10.0.0.3 with 32 bytes of data:
> Reply from 10.0.0.3 (http://10.0.0.3): bytes=32 time Reply from 10.0.0.3 (http://10.0.0.3): bytes=32 time Reply from 10.0.0.3 (http://10.0.0.3): bytes=32 time Reply from 10.0.0.3 (http://10.0.0.3): bytes=32 time
> C:UsersAdministrator>ping 10.0.0.254
>
> Pinging 10.0.0.254 with 32 bytes of data:
> Reply from 10.0.0.3 (http://10.0.0.3): Destination host unreachable.
> Reply from 10.0.0.3 (http://10.0.0.3): Destination host unreachable.
> Reply from 10.0.0.3 (http://10.0.0.3): Destination host unreachable.
> Reply from 10.0.0.3 (http://10.0.0.3): Destination host unreachable.
>
> C:UsersAdministrator>arp -a
>
> Interface: 10.0.0.3 --- 0xc
> Internet Address Physical Address Type
> 10.0.0.255 ff-ff-ff-ff-ff-ff static
> 224.0.0.22 01-00-5e-00-00-16 static
> 224.0.0.252 01-00-5e-00-00-fc static
>
> (PF log testing)
> ================
>
> I was curious to see if PF was seeing packets on the bridge interface, so I added the following to /etc/pf.conf:
> pass log (all) all
>
> I then ran the following command, which had no results when I tried a test ping:
> sudo tcpdump -n -e -ttt -i pflog0 host 10.0.0.3
>
> (I also watched live traffic on 10.0.0.254 and saw nothing from this IP)
>
>
> Summary
> ========
>
> I appear to be having some network connectivity issues that either lies in bhyve, virtio-net or the winvirtio drivers. If anybody has any suggestions for things I could try I'd love to hear it.
>
> Thanks,
>
> Jonathan.
> _______________________________________________
> freebsd-virtualization at freebsd.org (mailto:freebsd-virtualization at freebsd.org) mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization (https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization)
> To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe at freebsd.org (mailto:freebsd-virtualization-unsubscribe at freebsd.org)" 

	I had to disable offloading on the host NIC to get the network working inside bhyve session. This seems to be a FreeBSD bug in virtio stack. 

	ifconfig_igb0="-rxcsum -txcsum -vlanmtu -vlanhwtso -tso4 -tso6 -vlanhwtso -vlanhwcsum up" 

	-Rajil