Re: epair and vnet jail loose connection.

From: Michael Gmelin <grembo_at_freebsd.org>
Date: Sat, 12 Mar 2022 20:00:13 UTC
Hi Johan,

I created a setup similar to yours - so it's based on your config
files, but I had to tweak a few things to make it work.

Basically this what it looks like:

Host: Bare metal
Runs bhyve, bridge has 10.1.1.1, does NAT to the world

Bhyve VM runs 14-CURRENT (latest snapshot and has).
  vtnet0
  - 10.1.1.16/24
  - 10.1.1.17/32
  bridge0:
  - ip: 10.233.185.1/24
  - members: epair18a, epair20a
  jails:
  - haproxy.test.nl
    ip: 10.233.185.20
    haproxy: listens on 443 (alpn h2,http/1.1 tls), backend to
             10.233.185.18 port 80
    binat: 10.1.1.16
  - web01.test.nl
    ip: 10.233.185.18
    nginx: listens on 80 and serves the static page from your example
    binat: 10.1.1.17
  
Runing hey on this setup from the bare metal host that hosts the bhyve
vm works ok:

    # hey -h2 -n 10 -c 10 -z 300s https://10.1.1.16
    Summary:
      Total:        300.0030 secs
      Slowest:      5.0101 secs
      Fastest:      0.0013 secs
      Average:      0.0039 secs
      Requests/sec: 2582.7142
      Total data:   110024724 bytes       

On Sat, 12 Mar 2022 15:18:38 +0100
Johan Hendriks <joh.hendriks@gmail.com> wrote:
> ....

> mount.devfs;
> sysvshm="new";
> sysvsem="new";
> allow.raw_sockets;
> allow.set_hostname = 0;
> allow.sysvipc;
> enforce_statfs = "2";
> devfs_ruleset     = "11";

What is in devfs_ruleset 11? (it's not a standard one), I used "4" in
my tests.

> 
> path = "/storage/jails/${name}";
> host.hostname = "${name}.${domain}";
> 
> 
> web01 {
>     $ip = 18;
> }

I changed web01 to be the same setup as haproxy (that is, a full jail
based in /storage/jails/${name}), as I didn't really know how it worked
in your setup.

> 
> haproxy {
>     $ip = 20;
>     mount.fstab = "";
>     path = "/storage/jails/${name}";
> }

Best
Michael


-- 
Michael Gmelin