From nobody Wed Aug 17 20:22:45 2022 X-Original-To: freebsd-net@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4M7KFk5F77z4ZB84 for ; Wed, 17 Aug 2022 20:22:50 +0000 (UTC) (envelope-from grembo@freebsd.org) Received: from mail.evolve.de (mail.evolve.de [213.239.217.29]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA512 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mail.evolve.de", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4M7KFj2RC3z3D8f; Wed, 17 Aug 2022 20:22:49 +0000 (UTC) (envelope-from grembo@freebsd.org) Received: by mail.evolve.de (OpenSMTPD) with ESMTP id e610dc84; Wed, 17 Aug 2022 20:22:47 +0000 (UTC) Received: by mail.evolve.de (OpenSMTPD) with ESMTPSA id 5664317e (TLSv1.3:AEAD-AES256-GCM-SHA384:256:NO); Wed, 17 Aug 2022 20:22:47 +0000 (UTC) Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable List-Id: Networking and TCP/IP with FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-net List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-net@freebsd.org Mime-Version: 1.0 (1.0) Subject: Re: Tunnel interfaces and vnet boundary crossing From: Michael Gmelin In-Reply-To: <20220815085303.2c5cdb02@zeta.dino.sk> Date: Wed, 17 Aug 2022 22:22:45 +0200 Cc: freebsd-net@freebsd.org Message-Id: References: <20220815085303.2c5cdb02@zeta.dino.sk> To: Milan Obuch X-Mailer: iPhone Mail (19G71) X-Rspamd-Queue-Id: 4M7KFj2RC3z3D8f X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=softfail (mx1.freebsd.org: 213.239.217.29 is neither permitted nor denied by domain of grembo@freebsd.org) smtp.mailfrom=grembo@freebsd.org X-Spamd-Result: default: False [-2.60 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; MV_CASE(0.50)[]; MIME_GOOD(-0.10)[text/plain]; RCPT_COUNT_TWO(0.00)[2]; MLMMJ_DEST(0.00)[freebsd-net@freebsd.org]; ARC_NA(0.00)[]; R_DKIM_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; ASN(0.00)[asn:24940, ipnet:213.239.192.0/18, country:DE]; MIME_TRACE(0.00)[0:+]; TO_MATCH_ENVRCPT_SOME(0.00)[]; R_SPF_SOFTFAIL(0.00)[~all:c]; RCVD_TLS_LAST(0.00)[]; FREEFALL_USER(0.00)[grembo]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; FROM_HAS_DN(0.00)[]; DMARC_NA(0.00)[freebsd.org]; TO_DN_SOME(0.00)[]; MID_RHS_MATCH_FROM(0.00)[] X-ThisMailContainsUnwantedMimeParts: N > On 15. Aug 2022, at 08:52, Milan Obuch wrote: >=20 > =EF=BB=BFHi, >=20 > some time ago I managed to design and implement multi-tenant OpenVPN > server using vnet jails. This way I am able to use more OpenVPN > instances on single public IP. >=20 > This is made possible using tun/tap interface property allowing to > cross vnet boundary - here is part of my initialisation command sequence > for one instance: >=20 > jail -c name=3Dov1 vnet persist > jexec ov1 hostname -s ov1 > jexec ov1 ifconfig lo0 127.0.0.1/8 > jexec ov1 sysctl net.inet.ip.forwarding=3D1 > ifconfig tun1 create vnet ov1 > /usr/local/sbin/openvpn --cd /usr/local/etc/openvpn --daemon ov1 --config o= v1.cfg --writepid /var/run/ov1.pid >=20 > In ov1.cfg, relevant bits are >=20 > port 1001 > management localhost 2001 > dev tun1 >=20 > (Actual numbers are different, but important thing is how they relate > together.) >=20 > This way, OpenVPN process runs in base vnet, using one side of > pre-created tun/tap interface, while networking uses the other side of > this interface in child vnet, isolated from base vnet (and other OpenVPN > instances as well). >=20 > Presently, I am using vlan interfaces on one ethernet interface to > connect individual instances to their respective local network. I'd > like to replace this with some tunnel interface (gif, gre, ideally > ipsec secured). The best way to illustrate is using Cisco config > snippet: >=20 > interface Tunnel1 > vrf forwarding vrf1 > ip address 192.168.0.1 255.255.255.252 > tunnel source Loopback0 > tunnel destination 172.16.0.1 >=20 > This means outer layer uses base route table for tunnel creation, while > inner layer, packets/datagrams transferred over tunnel, use other vrf. >=20 > I tried to mimic this in FreeBSD with following commands: >=20 > ifconfig gre1 create tunnel 172.16.1.1 172.16.0.1 vnet ov1 > jexec ov1 ifconfig gre1 10.1.0.2/30 10.1.0.1 >=20 > This does not work. I found some older post which made me believing > this is caused by clearing whole tunnel configuration after moving > interface into different vnet. My (failed) tests indicate this is most > probably the cause. >=20 > So, my question is, does anybody use tunnel interface similar way? Is > it possible to achieve what I am trying with netgraph? I am able to > create some inter-vnet link using epair interface, but this is > something different. Or ideally, is somebody using IPSEC with VNET > jails, processing encapsulating packets in base and raw content in some > child vnet? >=20 Not sure if that helps you at all, but what I=E2=80=99ve done in the past is= create a tunnel interface on the jailhost and add a devfs rule to allow acc= ess to it from within the vnet jail. I then run OpenVPN within that jail (so= OpenVPN and tunnel interface are in the same jail). It=E2=80=99s super stable, only issue is that you need to be careful when to= release/destroy the interface on jail restart, otherwise it will become una= vailable on the jailhost and in a (new) jail. Best Michael