From nobody Fri Jul 23 16:36:25 2021 X-Original-To: freebsd-jail@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 53D8B12D14B9 for ; Fri, 23 Jul 2021 16:36:38 +0000 (UTC) (envelope-from jacques+freebsd@foucry.net) Received: from mail.foucry.net (fournil.foucry.net [95.217.83.231]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4GWZhj1mMTz3Jss for ; Fri, 23 Jul 2021 16:36:36 +0000 (UTC) (envelope-from jacques+freebsd@foucry.net) Received: from mail.foucry.net (unknown [192.168.12.17]) by mail.foucry.net (Postfix) with ESMTP id 44328F130 for ; Fri, 23 Jul 2021 16:36:29 +0000 (UTC) X-Virus-Scanned: amavisd-new at foucry.net Received: from mail.foucry.net ([192.168.12.17]) by mail.foucry.net (mail.foucry.net [192.168.12.17]) (amavisd-new, port 10024) with ESMTP id UJjdfThmi4kM for ; Fri, 23 Jul 2021 16:36:28 +0000 (UTC) Received: by mail.foucry.net (Postfix, from userid 58) id 65BA9EF1F; Fri, 23 Jul 2021 16:36:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=foucry.net; s=dkim; t=1627058188; bh=qepLsvCAqHt9S6DO92vmcCGWR/pnSph6LnuOL9gd4uk=; h=Date:From:To:Subject; b=IiwX0t1i5MM3meenJrufM6+wx7XSAqBl2OPAyDzSKOnJF7UyryyyPWBnyihZpek3e 1WSrLych9pdAPt6huVanmGPq2xvxuYx7x78MyD5vL8naKb78HR/vh6HTwO4vo5WUaX QVxDdEPZY6qLnkFJhj/inhWt8RbiVnUJKVHV08Hs= Received: from mithril.foucry.net (unknown [IPv6:2a01:e0a:434:44e0:ea6a:64ff:fe07:95a1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mail.foucry.net (Postfix) with ESMTPSA id 78683EF1E; Fri, 23 Jul 2021 16:36:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=foucry.net; s=dkim; t=1627058187; bh=qepLsvCAqHt9S6DO92vmcCGWR/pnSph6LnuOL9gd4uk=; h=Date:From:To:Subject; b=Ju9FgXlcg/kEkK69c4BtN8Ekz0Ke3PW0iCOfe/s4ePe5GEzn9rb2Q+mx/0RiBCnKS 4DwHIKOQDzXYN7egEWBMdl7OthJi6nSEaLF6qILc1y2wu3PMlWc2wVwu0N/kygHvgk EuYbBo1dAMn8GG97fbtw2NwHmbB30TR6f+Rx+UzA= Received: from mithril.foucry.net (localhost [IPv6:::1]) by mithril.foucry.net (Postfix) with ESMTPS id BE7A71098; Fri, 23 Jul 2021 18:36:26 +0200 (CEST) Date: Fri, 23 Jul 2021 18:36:25 +0200 From: Jacques Foucry To: freebsd-questions@freebsd.org, freebsd-jail@freebsd.org Subject: iocage, vnet jail does not go outside Message-ID: Mail-Followup-To: freebsd-questions@freebsd.org, freebsd-jail@freebsd.org List-Id: Discussion about FreeBSD jail(8) List-Archive: https://lists.freebsd.org/archives/freebsd-jail List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-jail@freebsd.org MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="08+5bsy7v9+aNml6" Content-Disposition: inline X-Operating-System: FreeBSD X-Rspamd-Queue-Id: 4GWZhj1mMTz3Jss X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=fail (headers rsa verify failed) header.d=foucry.net header.s=dkim header.b=IiwX0t1i; dkim=fail (headers rsa verify failed) header.d=foucry.net header.s=dkim header.b=Ju9FgXlc; dmarc=pass (policy=none) header.from=foucry.net; spf=pass (mx1.freebsd.org: domain of jacques@foucry.net designates 95.217.83.231 as permitted sender) smtp.mailfrom=jacques@foucry.net X-Spamd-Result: default: False [-5.85 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(-0.20)[+mx]; R_DKIM_REJECT(0.00)[foucry.net:s=dkim]; TO_DN_NONE(0.00)[]; DKIM_TRACE(0.00)[foucry.net:-]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(0.00)[foucry.net,none]; NEURAL_HAM_SHORT(-0.95)[-0.946]; DMARC_POLICY_ALLOW_WITH_FAILURES(-0.50)[]; SIGNED_PGP(-2.00)[]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[95.217.83.231:from]; ASN(0.00)[asn:24940, ipnet:95.217.0.0/16, country:DE]; MIME_TRACE(0.00)[0:+,1:+,2:~]; TAGGED_FROM(0.00)[freebsd]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; RCVD_COUNT_FIVE(0.00)[6]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.20)[multipart/signed,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-jail@freebsd.org]; SPAMHAUS_ZRD(0.00)[95.217.83.231:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MAILMAN_DEST(0.00)[freebsd-jail] X-ThisMailContainsUnwantedMimeParts: N --08+5bsy7v9+aNml6 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hello friends, I'm turing crazy. I made a new jail ,on my hosted system using iocage. Here is the config.json file: more config.json { "allow_mount": 1, "allow_mount_devfs": 1, "allow_mount_nullfs": 1, "allow_mount_procfs": 1, "allow_mount_tmpfs": 1, "allow_mount_zfs": 1, "allow_raw_sockets": 1, "allow_socket_af": 1, "allow_sysvipc": 1, "bpf": 1, "cloned_release": "13.0-RELEASE", "defaultrouter": "10.0.10.1", "defaultrouter6": "auto", "dhcp": 0, "host_hostname": "examplejail", "host_hostuuid": "examplejail", "ip4_addr": "vnet0|10.0.10.23/24", "ip6_addr": "vnet0|2a01:4f9:4a:1fd8::23", "jail_zfs_dataset": "iocage/jails/examplejail/data", "last_started": "2021-07-23 15:11:28", "nat": 0, "release": "13.0-RELEASE-p3", "vnet": 1, "vnet0_mac": "b42e999c5bca b42e999c5bcb", "vnet_default_interface": "auto" } The jail's ifconfig: ifconfig lo0: flags=3D8049 metric 0 mtu 16384 options=3D680003 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1 inet 127.0.0.1 netmask 0xff000000 groups: lo nd6 options=3D21 pflog0: flags=3D0<> metric 0 mtu 33160 groups: pflog epair0b: flags=3D8843 metric 0 mtu = 1500 options=3D8 ether b4:2e:99:9c:5b:cb hwaddr 02:ae:46:07:62:0b inet 10.0.10.23 netmask 0xffffff00 broadcast 10.0.10.255 inet6 2a01:4f9:4a:1fd8::23 prefixlen 64 inet6 fe80::b62e:99ff:fe9c:5bcb%epair0b prefixlen 64 scopeid 0x3 groups: epair media: Ethernet 10Gbase-T (10Gbase-T ) status: active nd6 options=3D21 The jail's netstat: netstat -rn Routing tables Internet: Destination Gateway Flags Netif Expire default 10.0.10.1 UGS epair0b 10.0.10.0/24 link#3 U epair0b 10.0.10.23 link#3 UHS lo0 127.0.0.1 link#1 UH lo0 Internet6: Destination Gateway Flags N= etif Expire ::/96 ::1 UGRS = lo0 default fe80::1%epair0b UGS epa= ir0b ::1 link#1 UHS = lo0 ::ffff:0.0.0.0/96 ::1 UGRS = lo0 2a01:4f9:4a:1fd8::/64 link#3 U epa= ir0b 2a01:4f9:4a:1fd8::23 link#3 UHS = lo0 fe80::/10 ::1 UGRS = lo0 fe80::%lo0/64 link#1 U = lo0 fe80::1%lo0 link#1 UHS = lo0 fe80::%epair0b/64 link#3 U epa= ir0b fe80::b62e:99ff:fe9c:5bcb%epair0b link#3 UHS = lo0 ff02::/16 On the host, the ifconfig (note thereis a lot of old fashion jails): ifconfig em0: flags=3D8963 metric 0 = mtu 1500 options=3D4810099 ether b4:2e:99:6a:80:9d inet6 2a01:4f9:4a:1fd8::2 prefixlen 64 inet6 fe80::b62e:99ff:fe6a:809d%em0 prefixlen 64 scopeid 0x1 inet6 2a01:4f9:4a:1fd8::5 prefixlen 64 inet6 2a01:4f9:4a:1fd8::11 prefixlen 64 inet6 2a01:4f9:4a:1fd8::12 prefixlen 64 inet6 2a01:4f9:4a:1fd8::15 prefixlen 64 inet6 2a01:4f9:4a:1fd8::16 prefixlen 64 inet6 2a01:4f9:4a:1fd8::18 prefixlen 64 inet6 2a01:4f9:4a:1fd8::19 prefixlen 64 inet6 2a01:4f9:4a:1fd8::21 prefixlen 64 inet6 2a01:4f9:4a:1fd8::22 prefixlen 64 inet6 2a01:4f9:4a:1fd8::25 prefixlen 64 inet6 2a01:4f9:4a:1fd8::14 prefixlen 64 inet6 2a01:4f9:4a:1fd8::29 prefixlen 64 inet6 2a01:4f9:4a:1fd8::17 prefixlen 64 inet 95.217.83.231 netmask 0xffffffc0 broadcast 95.217.83.255 media: Ethernet autoselect (1000baseT ) status: active nd6 options=3D21 lo0: flags=3D8049 metric 0 mtu 16384 options=3D680003 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2 inet 127.0.0.1 netmask 0xff000000 inet 127.0.12.1 netmask 0xff000000 inet 127.0.1.5 netmask 0xffffffff inet 127.0.1.11 netmask 0xffffffff inet 127.0.1.12 netmask 0xffffffff inet 127.0.1.15 netmask 0xffffffff inet 127.0.1.16 netmask 0xffffffff inet 127.0.1.18 netmask 0xffffffff inet 127.0.1.19 netmask 0xffffffff inet 127.0.1.21 netmask 0xffffffff inet 127.0.1.22 netmask 0xffffffff inet 127.0.1.25 netmask 0xffffffff inet 127.0.1.14 netmask 0xffffffff inet 127.0.1.29 netmask 0xffffffff inet 127.0.1.17 netmask 0xffffffff groups: lo nd6 options=3D21 lo1: flags=3D8049 metric 0 mtu 16384 options=3D680003 inet 192.168.12.1 netmask 0xffffff00 inet 192.168.12.5 netmask 0xffffffff inet 192.168.12.11 netmask 0xffffff00 inet 192.168.12.12 netmask 0xffffff00 inet 192.168.12.15 netmask 0xffffff00 inet 192.168.12.16 netmask 0xffffff00 inet 192.168.12.18 netmask 0xffffff00 inet 192.168.12.19 netmask 0xffffff00 inet 192.168.12.21 netmask 0xffffff00 inet 192.168.12.22 netmask 0xffffff00 inet 192.168.12.25 netmask 0xffffff00 inet 192.168.12.14 netmask 0xffffff00 inet 192.168.12.29 netmask 0xffffff00 inet 192.168.12.17 netmask 0xffffff00 groups: lo nd6 options=3D29 pflog0: flags=3D100 metric 0 mtu 33160 groups: pflog bridge0: flags=3D8843 metric 0 mtu = 1500 description: jails-bridge ether 58:9c:fc:10:ed:66 inet 10.0.10.1 netmask 0xffffff00 broadcast 10.0.10.255 id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15 maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200 root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0 member: vnet0.655 flags=3D143 ifmaxaddr 0 port 6 priority 128 path cost 2000 member: em0 flags=3D143 ifmaxaddr 0 port 1 priority 128 path cost 20000 groups: bridge nd6 options=3D9 vnet0.655: flags=3D8943 met= ric 0 mtu 1500 description: associated with jail: examplejail as nic: epair0b options=3D8 ether b4:2e:99:9c:5b:ca hwaddr 02:ae:46:07:62:0a groups: epair media: Ethernet 10Gbase-T (10Gbase-T ) status: active nd6 options=3D29 And host's netstat (again with many old fashion jail): netstat -rn Routing tables Internet: Destination Gateway Flags Netif Expire default 95.217.83.193 UGS em0 10.0.10.0/24 link#5 U bridge0 10.0.10.1 link#5 UHS lo0 95.217.83.192/26 link#1 U em0 95.217.83.231 link#1 UHS lo0 127.0.0.1 link#2 UH lo0 127.0.1.5 link#2 UH lo0 127.0.1.11 link#2 UH lo0 127.0.1.12 link#2 UH lo0 127.0.1.14 link#2 UH lo0 127.0.1.15 link#2 UH lo0 127.0.1.16 link#2 UH lo0 127.0.1.17 link#2 UH lo0 127.0.1.18 link#2 UH lo0 127.0.1.19 link#2 UH lo0 127.0.1.21 link#2 UH lo0 127.0.1.22 link#2 UH lo0 127.0.1.25 link#2 UH lo0 127.0.1.29 link#2 UH lo0 127.0.12.1 link#2 UH lo0 192.168.12.1 link#3 UH lo1 192.168.12.5 link#3 UH lo1 192.168.12.11 link#3 UH lo1 192.168.12.12 link#3 UH lo1 192.168.12.14 link#3 UH lo1 192.168.12.15 link#3 UH lo1 192.168.12.16 link#3 UH lo1 192.168.12.17 link#3 UH lo1 192.168.12.18 link#3 UH lo1 192.168.12.19 link#3 UH lo1 192.168.12.21 link#3 UH lo1 192.168.12.22 link#3 UH lo1 192.168.12.25 link#3 UH lo1 192.168.12.29 link#3 UH lo1 Internet6: Destination Gateway Flags N= etif Expire ::/96 ::1 UGRS = lo0 default fe80::1%em0 UGS = em0 ::1 link#2 UHS = lo0 ::ffff:0.0.0.0/96 ::1 UGRS = lo0 2a01:4f9:4a:1fd8::/64 link#1 U = em0 2a01:4f9:4a:1fd8::2 link#1 UHS = lo0 2a01:4f9:4a:1fd8::5 link#1 UHS = lo0 2a01:4f9:4a:1fd8::11 link#1 UHS = lo0 2a01:4f9:4a:1fd8::12 link#1 UHS = lo0 2a01:4f9:4a:1fd8::14 link#1 UHS = lo0 2a01:4f9:4a:1fd8::15 link#1 UHS = lo0 2a01:4f9:4a:1fd8::16 link#1 UHS = lo0 2a01:4f9:4a:1fd8::17 link#1 UHS = lo0 2a01:4f9:4a:1fd8::18 link#1 UHS = lo0 2a01:4f9:4a:1fd8::19 link#1 UHS = lo0 2a01:4f9:4a:1fd8::21 link#1 UHS = lo0 2a01:4f9:4a:1fd8::22 link#1 UHS = lo0 2a01:4f9:4a:1fd8::25 link#1 UHS = lo0 2a01:4f9:4a:1fd8::29 link#1 UHS = lo0 fe80::/10 ::1 UGRS = lo0 fe80::%em0/64 link#1 U = em0 fe80::b62e:99ff:fe6a:809d%em0 link#1 UHS = lo0 fe80::%lo0/64 link#2 U = lo0 fe80::1%lo0 link#2 UHS = lo0 ff02::/16 ::1 UGRS = lo0 The bridge0 had the em0 and vnet0:655 interfaces. =46rom the jail in can ping oustside world: ping google.ca PING6(56=3D40+8+8 bytes) 2a01:4f9:4a:1fd8::23 --> 2a00:1450:400f:803::2003 16 bytes from 2a00:1450:400f:803::2003, icmp_seq=3D0 hlim=3D118 time=3D7.92= 7 ms 16 bytes from 2a00:1450:400f:803::2003, icmp_seq=3D1 hlim=3D118 time=3D7.80= 0 ms 16 bytes from 2a00:1450:400f:803::2003, icmp_seq=3D2 hlim=3D118 time=3D7.79= 8 ms ^C --- google.ca ping6 statistics --- 3 packets transmitted, 3 packets received, 0.0% packet loss round-trip min/avg/max/std-dev =3D 7.798/7.842/7.927/0.061 ms The problem is, I cannot ssh to an external computer (for example, my nextcloud hosted at home): ssh -vvv nextcloud.foucry.net -p2250 OpenSSH_7.9p1, OpenSSL 1.1.1k-freebsd 25 Mar 2021 debug1: Reading configuration data /etc/ssh/ssh_config debug2: resolving "nextcloud.foucry.net" port 2250 debug2: ssh_connect_direct debug1: Connecting to nextcloud.foucry.net [2a01:e0a:434:44e0:ff:60ff:feba:= b582] port 2250. debug1: connect to address 2a01:e0a:434:44e0:ff:60ff:feba:b582 port 2250: O= peration timed out debug1: Connecting to nextcloud.foucry.net [82.65.174.130] port 2250. debug1: connect to address 82.65.174.130 port 2250: Operation timed out ssh: connect to host nextcloud.foucry.net port 2250: Operation timed out What's look strange (for me) is the traceroute (using ipv4): traceroute nextcloud.foucry.net traceroute to nextcloud.foucry.net (82.65.174.130), 64 hops max, 40 byte pa= ckets 1 10.0.10.1 (10.0.10.1) 0.086 ms 0.051 ms 0.037 ms 2 static.193.83.217.95.clients.your-server.de (95.217.83.193) 0.451 ms = 0.571 ms 0.392 ms 3 core32.hel1.hetzner.com (213.239.252.97) 11.621 ms core31.hel1.hetzner.com (213.239.252.93) 1.812 ms core32.hel1.hetzner.com (213.239.252.97) 2.793 ms 4 core9.fra.hetzner.com (213.239.224.166) 21.295 ms core8.fra.hetzner.com (213.239.224.149) 20.730 ms core9.fra.hetzner.com (213.239.224.170) 20.333 ms 5 core4.fra.hetzner.com (213.239.245.85) 28.499 ms core4.fra.hetzner.com (213.239.224.177) 20.507 ms 22.850 ms 6 * * * 7 * * * 8 * * * 9 * * * 10 * * * 11 * * * 12 * * * 13 *^C Look's like something wrong on the way, but I could connect on the same host form any other jails. There is for me a mysterious behaviiors that I can't understand.=20 Any help will be appreciate. Thanks for reading me, and the time your spend on my problem. --=20 Jacques Foucry --08+5bsy7v9+aNml6 Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iNUEAREKAH0WIQRd29C9s3PtOgNIX2tkcaT/7DX1XwUCYPrv/l8UgAAAAAAuAChp c3N1ZXItZnByQG5vdGF0aW9ucy5vcGVucGdwLmZpZnRoaG9yc2VtYW4ubmV0NURE QkQwQkRCMzczRUQzQTAzNDg1RjZCNjQ3MUE0RkZFQzM1RjU1RgAKCRBkcaT/7DX1 X4jGAQCG+zm53q9HlSsrWZffS3KWuSzdyKjqELP3Fr31Gt9WVAEAkwJZ2xsi+ZYA E7z13v6eK7+BTVkoGqzULIZSeTkO9XY= =jIaX -----END PGP SIGNATURE----- --08+5bsy7v9+aNml6--