NATed or Private Network Setups

[John Baldwin]

On Friday, October 24, 2014 04:08:27 PM Pete Wright wrote:
> Hi All,
> Has anyone deployed bhyve using NAT'd or private network setups?  I've
> been able to deploy bridged interfaces, but I was wondering if anyone
> has done other network topologies.  Is there anything preventing this
> from happening code wise?  I reckon it could be achieved by creating a
> pseudo interface?

I setup a bridge on my laptop and add all the tap interfaces for VMs as 
members to the bridge.  I use a /24 for the internal "LAN" for these 
interfaces and assign the .1 to the bridge0 interface itself.  I then run 
dnsmasq to provide DHCP/DNS to the VMs and use natd (ipfw_nat would also work) 
to allow the VMs NAT access to the outside world.  There are more details in 
an article in the most recent issue of the FreeBSD Journal, but I'll push that 
into the regular FreeBSD docs at some point as well.

With the dnsmasq setup, I put the vmname as the hostname so that it is sent in 
the dhclient request.  dnsmasq then adds local overrides for VMs while they 
are active.  (So you can 'ssh vm0' on the host, or from another vm.)  The 
'host' entry in /etc/hosts is also snarfed up by dnsmasq so that within a vm I 
can use 'host' as a hostname (e.g. for NFS mounting something off of my 
laptop).

Some config file snippets:

/etc/sysctl.conf:

net.link.tap.up_on_open=1

/etc/rc.conf:

# bhyve setup
autobridge_interfaces="bridge0"
autobridge_bridge0="tap*"
cloned_interfaces="bridge0 tap0 tap1 tap2"
ifconfig_bridge0="inet 192.168.16.1/24"
gateway_enable="YES"
natd_enable="YES"
natd_interface="wlan0"
dnsmasq_enable="YES"
firewall_enable="YES"
firewall_type="/etc/rc.firewall.pippin"

/etc/hosts:

192.168.16.1            host

/etc/resolvconf.conf:
 
name_servers=127.0.0.1
dnsmasq_conf=/etc/dnsmasq-conf.conf
dnsmasq_resolv=/etc/dnsmasq-resolv.conf

/usr/local/etc/dnsmasq.conf:

domain-needed
bogus-priv
resolv-file=/etc/dnsmasq-resolv.conf
interface=bridge0
dhcp-range=192.168.16.10,192.168.16.200,12h
conf-file=/etc/dnsmasq-conf.conf

/etc/rc.firewall.pippin:

# prevent inbound traffic for our guest /24
add     deny    all from any to 192.168.16.0/24 via em0
add     deny    all from any to 192.168.16.0/24 via wlan0

# divert packets between guest and outside world to natd
add     divert  natd all from any to any via wlan0

# prevent outbound traffic for our guest /24
add     deny    all from 192.168.16.0/24 to any via em0
add     deny    all from 192.168.16.0/24 to any via wlan0

# pass everything else
add     allow   all from any to any

(I have not figured out a way to have the NAT prefer em0 if present and fail 
over to wlan0 if not, etc.)

-- 
John Baldwin