From nobody Sat May 29 15:59:49 2021 X-Original-To: freebsd-jail@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 72992BFA83F; Sat, 29 May 2021 15:59:52 +0000 (UTC) (envelope-from mj-mailinglist@gmx.de) Received: from mout.gmx.net (mout.gmx.net [212.227.17.21]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "mout.gmx.net", Issuer "TeleSec ServerPass Class 2 CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4FsmTg4KHrz3kDy; Sat, 29 May 2021 15:59:51 +0000 (UTC) (envelope-from mj-mailinglist@gmx.de) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.net; s=badeba3b8450; t=1622303989; bh=FIG5t7pq+3T9XrI2mfFjEWOJEGQJxc1t3FPm+QW1ZHs=; h=X-UI-Sender-Class:From:To:Subject:Date; b=ZtkT6Q+rn1M7jTqvMyu4ndxq3/SGeIA3AE9AjaWN0/ec0dMZSZT07pF06sqVaFfGY RqT/YcLSbLkemqnT1sjnUAC8Ds3OaSrc6qtsC2cVNpHBfgZ/MCS119CZg9OeFt5q5M eJ5KcKmSrm3/ZcUs2L+AhtRguq1BW3ZwvmF9oAEk= X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c Received: from [89.245.246.156] ([89.245.246.156]) by web-mail.gmx.net (3c-app-gmx-bap59.server.lan [172.19.172.129]) (via HTTP); Sat, 29 May 2021 17:59:49 +0200 List-Id: Discussion about FreeBSD jail(8) List-Archive: https://lists.freebsd.org/archives/freebsd-jail List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-jail@freebsd.org MIME-Version: 1.0 Message-ID: From: mj-mailinglist@gmx.de To: freebsd-jail@freebsd.org, freebsd-current@freebsd.org Subject: Network in VNET jail does not work on my FreeBSD current bhyve vm Content-Type: text/plain; charset=UTF-8 Date: Sat, 29 May 2021 17:59:49 +0200 Importance: normal Sensitivity: Normal X-Priority: 3 X-Provags-ID: V03:K1:bJOKm1w6Ra+jn5x/7CelHq7lVaNl3Z/2CmukUgLWNKv88o34CBL51VDkf/wa0exc72nOV 7k7eF5yAG/4TAIlkA9rHxyqxLdZW+LOM+TjV3TBBMLJOEqaitxieIFUwDuGBNFWRXPvP1KhtFhhy BfSRUM0p5cOSmBLn79QrlA32iW0nIje1N+gxVrAVqrCb5FeA5oKk22Oast6+pk5fe/RGBRw3PvfI c29PlM3zjGcKzTRuxsQkBqERcRqzwS22T33J5Ep5gwflk13Gcg9NhMywpWJkTXAUy5t4awh/wJJB sw= X-Spam-Flag: NO X-UI-Out-Filterresults: notjunk:1;V03:K0:BMr66DBcBak=:g9pzPIjHmd0pPpBStdgPoV /4LvXG7NPmfVx9DoDX7WT0oHb6i2xgAS/PclXSEhG+6KqYCsTzP9tqDUcsEm3wGN0YtLAZZbd L25CUijAN/pqTdpb+zbIOI6K00ukv+bVWEAVq3NTu510KQmM/QNHB2jFOiq9P6Is4AG6ybf16 gdyUAPg+BmM3Ec1oPRBQ/8chzvL1b961kl8AvmPA7bvxr+UEx6U81L0TFCEKifNVyxgMEiXHV 6xCrhG7AKOo/zUE3AOzGdA8f2XBxQ+7mVXSr1rMMPF/dy+j4nmd2sVCMMFyuLzSG3bAmaypL+ VrA9ed0fPWPjtxlnN6zeaPMj+YAtSNoa/efGISCu5NVsQ3a5VLmhJ5sjsibEDN1ORNRH5xQSW TTJ7DuOMGVjtdOzT9nsjJEWVBkKijn3FQ7e3SG7nn67tvkJ+43TMA2tgjP9zVCWJP/Z4R3JOp 0P53+j97C6afu/e3oYHH8CYEY5wi8KOB7Dj1ewVj+8VgTAiOv821rMbjK9nXBB4Ed+p05AeQZ ZlkACKKQCxIECUmwIe3ppfrzKWBWofseL24Jg+/fzWhSR6tjS13XIz4qlh77C8NVrl/ex1mXH hcFfwNN0yBV2qWJ+MCD1PEzI/QYyR0Uy86MIRy57QdcWjwWaVQsDzEPv3sf987xwQEi3Csijj zCAuJHYKp1zltoH1TodMVgshaGha6FqOy39GOT5NIOQbXftmoXEafmn6lkvTdTq670R57wsUZ GImK+ZKXD1vDccdnsF30dbpEyvAXDW/28Woj9REJsEPpDjo0WbpiTDsXL5gpS8+KGCUYNGeRC CqdhYule42q/1iSVjd3qrxDLtLeUw67LiqCoAoGnQd+cxOQAPs/+peNtvH6iC3h5cEgnmRTZz j9tgBq5L2tTZaKRjrLmxE3nMXBBwB6vsJu8uMSNkFL7ilXm3BlbYimo7pMpXzWn+jhrlb6Usa HseLyqgrw5Q== X-Rspamd-Queue-Id: 4FsmTg4KHrz3kDy X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmx.net header.s=badeba3b8450 header.b=ZtkT6Q+r; dmarc=pass (policy=none) header.from=gmx.de; spf=pass (mx1.freebsd.org: domain of mj-mailinglist@gmx.de designates 212.227.17.21 as permitted sender) smtp.mailfrom=mj-mailinglist@gmx.de X-Spamd-Result: default: False [-3.60 / 15.00]; FREEMAIL_FROM(0.00)[gmx.de]; R_SPF_ALLOW(-0.20)[+ip4:212.227.17.0/27]; TO_DN_NONE(0.00)[]; DKIM_TRACE(0.00)[gmx.net:+]; RCPT_COUNT_TWO(0.00)[2]; HAS_X_PRIO_THREE(0.00)[3]; NEURAL_HAM_SHORT(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[gmx.de,none]; RECEIVED_SPAMHAUS_PBL(0.00)[89.245.246.156:received]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmx.de]; ASN(0.00)[asn:8560, ipnet:212.227.0.0/16, country:DE]; FROM_EQ_ENVFROM(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[gmx.net:s=badeba3b8450]; RCVD_IN_DNSWL_LOW(-0.10)[212.227.17.21:from]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; FROM_NO_DN(0.00)[]; MID_RHS_NOT_FQDN(0.50)[]; RWL_MAILSPIKE_VERYGOOD(0.00)[212.227.17.21:from]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-current,freebsd-jail] X-Spam: Yes X-ThisMailContainsUnwantedMimeParts: N Hello everybody, since a few weeks, my jails on a bhyve-vm, running current are not reachable via network, when configured with VNET. They can't even access the gateway. I don't remember when this problem started, but it's a few weeks. The same jail.conf works on a 13.0 host, on a current system the network does not work. A configuration without VNET on the same jail works. Are there any changes, that i missed? Here is the configuration, maybe someone spots an error, or has an idea what's going on: -- Martin uname on bhyve vm: ------------------ root@fbsd14:~ # uname -a FreeBSD fbsd14.fritz.box 14.0-CURRENT FreeBSD 14.0-CURRENT main-n247020-e0fa04e257c GENERIC-NODEBUG amd64 root@fbsd14:~ # freebsd-version -kru 14.0-CURRENT 14.0-CURRENT 14.0-CURRENT jail.conf on bhyve vm: ---------------------- # set default configuration values mount.devfs = true; exec.clean = true; allow.chflags = 1; allow.raw_sockets = 1; devfs_ruleset = 5; exec.system_user = "root"; exec.jail_user = "root"; exec.timeout = 30; stop.timeout = 30; ######### # Jails # ######### j1 { # Hostname host.hostname = "j1.fritz.box"; host.domainname = "fritz.box"; host.hostuuid = "68c2ad9b-b582-11eb-a925-589cfc0ac350"; osrelease = "14.0-CURRENT"; osreldate = "1400013"; # Network vnet = 1; vnet.interface = "epair2b"; exec.prestart += "ifconfig epair2 create up"; exec.prestart += "ifconfig epair2a description 'IFID=2 JAIL=j1'"; exec.prestart += "ifconfig bridge0 addm epair2a"; command = "ifconfig epair2b inet 192.168.1.101/22"; command += "route -n add -inet default 192.168.0.1"; exec.prestop = "ifconfig epair2b -vnet j1"; exec.poststop += "ifconfig bridge0 deletem epair2a"; exec.poststop += "ifconfig epair2a destroy"; sysvmsg = new; sysvsem = new; sysvshm = new; path = "/jails/j1"; allow.mount.zfs = 1; ## Script execution exec.timeout = 90; # Pre-/Post-Scripts exec.prestart += "logger trying to start jail j1 ..."; exec.poststart += "logger jail j1 has started"; exec.prestop += "logger shutting down jail j1"; exec.poststop += "logger jail j1 has shut down"; # Start Script exec.start = "/bin/sh /etc/rc"; exec.stop = "/bin/sh /etc/rc.shutdown"; } ----------------------------------- /etc/rc.conf on bhyve vm: ------------------------- syslogd_flags="-ss" sendmail_enable="NONE" hostname="fbsd14.fritz.box" ifconfig_vtnet0="inet 192.168.1.100 netmask 255.255.252.0" defaultrouter="192.168.0.1" local_unbound_enable="YES" sshd_enable="YES" ntpd_enable="YES" # Set dumpdev to "AUTO" to enable crash dumps, "NO" to disable dumpdev="AUTO" zfs_enable="YES" jail_enable="YES" keymap="de" cloned_interfaces="bridge0" ifconfig_bridge0="addm vtnet0 up" # NFS rpc_lockd_enable="YES" rpc_statd_enable="YES" nfs_client_enable="YES" nfsuserd_enable="YES" ------------------------------------- ifconfig on bhyve vm: --------------------- root@fbsd14:~ # ifconfig -f inet:cidr vtnet0: flags=8863 metric 0 mtu 1500 options=80028 ether 58:9c:fc:0a:c3:50 inet 192.168.1.100/22 broadcast 192.168.3.255 media: Ethernet autoselect (10Gbase-T ) status: active nd6 options=29 lo0: flags=8049 metric 0 mtu 16384 options=680003 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2 inet 127.0.0.1/8 groups: lo nd6 options=21 bridge0: flags=8843 metric 0 mtu 1500 ether 58:9c:fc:10:ff:bf id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15 maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200 root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0 member: epair2a flags=143 ifmaxaddr 0 port 4 priority 128 path cost 2000 groups: bridge nd6 options=9 epair2a: flags=8943 metric 0 mtu 1500 description: IFID=2 JAIL=j1 options=8 ether 02:b4:ee:59:b3:0a groups: epair media: Ethernet 10Gbase-T (10Gbase-T ) status: active nd6 options=29 ------------------------------- /etc/rc.conf in jail: --------------------- syslogd_flags="-ss" sendmail_enable="NO" sendmail_submit_enable="NO" sendmail_outbound_enable="NO" sendmail_msp_queue_enable="NO" sshd_enable="YES" --------------------------- ifconfig in jail: ----------------- root@j1:~ # ifconfig -f inet:cidr lo0: flags=8049 metric 0 mtu 16384 options=680003 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1 inet 127.0.0.1/8 groups: lo nd6 options=21 epair2b: flags=8843 metric 0 mtu 1500 options=8 ether 02:b4:ee:59:b3:0b inet 192.168.1.101/22 broadcast 192.168.3.255 groups: epair media: Ethernet 10Gbase-T (10Gbase-T ) status: active nd6 options=29 ------------------------------------ uname in jail: -------------- root@j1:~ # uname -a FreeBSD j1.fritz.box 14.0-CURRENT FreeBSD 14.0-CURRENT main-n247020-e0fa04e257c GENERIC-NODEBUG amd64 root@j1:~ # freebsd-version -ru 14.0-CURRENT 14.0-CURRENT