From nobody Tue Aug 23 18:06:59 2022 X-Original-To: freebsd-net@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4MBxyZ0cM7z4bBlg for ; Tue, 23 Aug 2022 18:07:18 +0000 (UTC) (envelope-from tt78347@protonmail.com) Received: from mail-4325.protonmail.ch (mail-4325.protonmail.ch [185.70.43.25]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "protonmail.com", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4MBxyY1cgWz3MKk; Tue, 23 Aug 2022 18:07:17 +0000 (UTC) (envelope-from tt78347@protonmail.com) Date: Tue, 23 Aug 2022 18:06:59 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=protonmail.com; s=protonmail3; t=1661278034; x=1661537234; bh=NcpgPCM7yTZIdkR3B8rZmR0ZL6FWtRlx2OtNOXw3/oo=; h=Date:To:From:Cc:Reply-To:Subject:Message-ID:In-Reply-To: References:Feedback-ID:From:To:Cc:Date:Subject:Reply-To: Feedback-ID:Message-ID; b=YEg8fVFIM2+Xax6+UzgaiMOTSnWx2Z57ZgH3oG6/PbmfY0NabPUVHCHb/WsBuMBkb ig75Ahm+Kpnh4BIIEbbJ65yGG6iuFlwd32yNy05SQraTpl5bNbZoU3MN50Qa5vSzCG ahKI6l9kIJ6DOhkrQylt/d1uk0ZzdfjpbxCGRo4wfqdaULDwL8OQ+/KF/jbhFAMYOT 6FIg1Nk0sA329Al0Gop0TG07pJpROAok7mrM/IcSv8qxAmA5FdR06ys1vY9Hn5QgKw 9AXdAgSiX6N4yg1R2d55u80XrMfzEmENEKSeeI2QS3ss2fCjlYbEsDP5ny6PiXeuXd twy77e3k72p8w== To: Milan Obuch From: tt78347 Cc: freebsd-net@freebsd.org, grembo@freebsd.org Reply-To: tt78347 Subject: Re: Tunnel interfaces and vnet boundary crossing Message-ID: In-Reply-To: <20220823175918.4696603f@zeta.dino.sk> References: <3N4WPlYG_OY2JAm4Rg5nWFjqo650j0ccOEOUWSwvWEp9cOudWPfFQ3C54QddWJcqyKd450QB7eczvghXmHbgImkn0tk2pAg7zbn49dJlPtM=@protonmail.com> <20220823175918.4696603f@zeta.dino.sk> Feedback-ID: 43781169:user:proton List-Id: Networking and TCP/IP with FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-net List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-net@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 4MBxyY1cgWz3MKk X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=pass header.d=protonmail.com header.s=protonmail3 header.b=YEg8fVFI; dmarc=pass (policy=quarantine) header.from=protonmail.com; spf=pass (mx1.freebsd.org: domain of tt78347@protonmail.com designates 185.70.43.25 as permitted sender) smtp.mailfrom=tt78347@protonmail.com X-Spamd-Result: default: False [0.39 / 15.00]; NEURAL_SPAM_MEDIUM(0.97)[0.969]; NEURAL_SPAM_LONG(0.94)[0.940]; NEURAL_HAM_SHORT(-0.52)[-0.523]; DMARC_POLICY_ALLOW(-0.50)[protonmail.com,quarantine]; R_DKIM_ALLOW(-0.20)[protonmail.com:s=protonmail3]; R_SPF_ALLOW(-0.20)[+ip4:185.70.43.0/24]; MIME_GOOD(-0.10)[text/plain]; MLMMJ_DEST(0.00)[freebsd-net@freebsd.org]; FROM_EQ_ENVFROM(0.00)[]; RCVD_COUNT_ZERO(0.00)[0]; FREEMAIL_ENVFROM(0.00)[protonmail.com]; MIME_TRACE(0.00)[0:+]; DKIM_TRACE(0.00)[protonmail.com:+]; HAS_REPLYTO(0.00)[tt78347@protonmail.com]; ASN(0.00)[asn:62371, ipnet:185.70.43.0/24, country:CH]; FROM_HAS_DN(0.00)[]; REPLYTO_EQ_FROM(0.00)[]; ARC_NA(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; FREEMAIL_REPLYTO(0.00)[protonmail.com]; FREEMAIL_FROM(0.00)[protonmail.com]; TO_DN_SOME(0.00)[]; MID_RHS_MATCH_FROM(0.00)[] X-ThisMailContainsUnwantedMimeParts: N ------- Original Message ------- On Tuesday, August 23rd, 2022 at 3:58 PM, Milan Obuch = wrote: > On Tue, 23 Aug 2022 15:06:00 +0000 > tt78347 tt78347@protonmail.com wrote: >=20 > > Hello. This is my first email to this mailing list, so I hope it has > > been sent and formatted correctly. >=20 >=20 > Well, top posting is being frowned upon a bit on mailing list, which > has its racio, otherwise I think everything is fine :) >=20 > > Regarding tun interfaces, I do something similar to M. Gmelin, > > however, I create the tunnel (tun, for openvpn) within the vnet jail. > > I think your question is not about tun interfaces but about gre, > > which I discuss near the end of this email. >=20 >=20 > You are right, terminology is sometimes somewhat ambiguous. I have no > issues with software tunnel interface (man tun). I did not find the > right configuration for my needs for network tunnels (man gif, man gre, > man if_ipsec, the last one could be the most desired one). >=20 > > Regarding devfs: > > My /etc/devfs.rules contains the following ruleset, applicable to all > > such jails: > >=20 > > [devfsrules_jail=3D5] > > add include $devfsrules_hide_all > > add include $devfsrules_unhide_basic > > add include $devfsrules_unhide_login > > add path bpf unhide > > add path tun* unhide > > add path bpf0 unhide > >=20 > > This ruleset ID is specified during jail creation. If you are > > modifying this on a running system, I think you will need to use the > > devfs utility. This is how I get tun interfaces into a jail, for use > > with openvpn or otherwise. >=20 >=20 > I use something similar in my setup, basically with expanded includes. >=20 > > General background: > > All the vnet jails are to be connected to a bridge with epairs, and > > NAT is to be performed for jail traffic exchanged with the > > exterior. >=20 >=20 > In general, epair is optional, in some use cases vnet jail could be > decisively insulated from the rest of jails. But I know this one and > use it if in accordance with my design goals. OK. I just meant it to specify what was included in my suggested design. >=20 > > There is an additional vnet jail which handles ipsec; the ipsec jail > > connects the host's jail network (usually a /24) to other local > > networks. Each external local network's route is specified in the > > host's routing table, with a next-hop being the ipsec jail. I > > personally don't use ipsec interfaces, so in the ipsec jail the > > traffic is extracted by the kernel (per TSs that are configured with > > StrongSwan), encapsulated and sent out the host's external interface. > > I have no knowledge of whether this is better or worse than using an > > ipsec interface. >=20 >=20 > This one is interested for me. Could you elaborate more on its design? > Output of 'ifconfig', 'netstat -rn', 'pkg info', 'cat /etc/rc.conf' > commands would be really usefull. If you do concern about privacy, > maybe edit it a bit (I am not interested in exact IPs and such info). I think I had misunderstood your original post, but here is information abo= ut this ipsec jail. I have replaced certain info with variable names. # cat /etc/rc.conf devfs_load_rulesets=3D"YES" hostname=3D"$HOSTNAME" syslogd_enable=3D"YES" syslogd_flags=3D"-s -v -v" inetd_enable=3D"NO" defaultrouter=3D"${HOST_NET}.1" firewall_enable=3D"YES" firewall_script=3D"/etc/ipfw.conf" firewall_logif=3D"YES" sendmail_enable=3D"NO"=20 sendmail_submit_enable=3D"NO"=20 sendmail_outbound_enable=3D"NO"=20 sendmail_msp_queue_enable=3D"NO" # cat /etc/ipfw.conf ipfw add allow ip from any to any # cat /etc/rc.conf.local=20 strongswan_enable=3D"YES" # netstat -rn Routing tables Internet: Destination Gateway Flags Netif Expire default ${HOST_NET}.1 UGS epair${JAIL_ID}b 127.0.0.1 link#1 UH lo0 ${HOST_NET}.0/24 link#${JAIL_ID} U epair${JAIL_ID}b ${HOST_NET}.${JAIL_ID} link#${JAIL_ID} UHS lo0 This ipsec jail has epair IP address of ${HOST_NET}.${JAIL_ID}.=20 For example, 192.168.1.100, where $JAIL_ID=3D100 and $HOST_NET=3D192.168.1 The host's bridge is at ${HOST_NET}.1 The only relevant package in this jail is security/strongswan. In this design, all vnet jails have addresses on $HOST_NET using epair inte= rfaces connected to one or more virtual host bridge interfaces. ARP takes place on this bridge, and the host forwards packets between jails= . The ipsec jail is just one of those jails.=20 The host also has routes for other sites' local networks. These routes are = set to go through the ipsec jail. The ipsec jail then encapsulates them, and from the host's perspective, whe= n it leaves the ipsec jail, it is just ESP/UDP. In this design, ipsec is used for site-to-site tunneling, and GRE is used f= or use-case-specific tunneling between hosts on this WAN. A common use case= for this is when a machine needs to set a route or a default route through= a gateway that is not on a connected subnet. Operating ipsec and GRE indep= endently means you can use GRE where necessary, with the whole WAN being en= capsulated with ipsec. Traffic within a host being unencrypted. >=20 > > Specific steps inside an openvpn jail: > >=20 > > After an openvpn jail is started, the tunnel (tun interface) is > > created in the jail by running the openvpn daemon. I think tunnel > > numbers (appearing on the interface name) need to be globally > > distinct. I'm not sure if you needed any assistance with this. >=20 >=20 > In my scenario, I use tun devices pre-created before running OpenVPN > process. This allows for better control and easier to understand setup. I think I misunderstood your posts. I thought you were running openvpn clie= nts, but I see from reading again that you are running openvpn servers. I h= ave never run an openvpn server before. >=20 > > Regarding gre interfaces, I also create these within the jail. I have > > not ever had any problems with this. I don't know that a specific > > device is needed, so I don't know if devfs is involved here. >=20 >=20 > This setup just requires connection to outer network, which I would > like specifically to be avoided as it makes setup a bit more > complicated. Just a reminder, I have almost twenty of running OpenVPN > instances in one jail, with almost twenty vnet jails handling the > routing for each VPN. I don't fully understand what is happening. What do you mean by outer netwo= rk, and why would it be an issue to create the GRE tunnel inside the jail, = instead of creating it outside the jail and moving it into the jail? I think you have your openvpn server instances in a vnet jail, and then chi= ld jails within that jail which are doing something related to routing. Are= you using many fwd-type firewall rules, instead of using default routes? Would it make sense to have each openvpn server in a separate vnet jail, ea= ch with its own GRE interface, and do away with your child vnet jails that = handle routing? Sorry for my misunderstandings. >=20 > [ snip ] >=20 > I hope this time I made it a bit clearer what I am looking for. >=20 > Regards, > Milan