From nobody Tue Nov 08 01:11:18 2022 X-Original-To: freebsd-net@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4N5qmt1mJzz4gw75 for ; Tue, 8 Nov 2022 01:11:26 +0000 (UTC) (envelope-from zlei.huang@gmail.com) Received: from mail-pg1-x531.google.com (mail-pg1-x531.google.com [IPv6:2607:f8b0:4864:20::531]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4N5qmr6Ngjz3RK2 for ; Tue, 8 Nov 2022 01:11:24 +0000 (UTC) (envelope-from zlei.huang@gmail.com) Received: by mail-pg1-x531.google.com with SMTP id o13so2757396pgu.7 for ; Mon, 07 Nov 2022 17:11:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=references:to:cc:in-reply-to:date:subject:mime-version:message-id :from:from:to:cc:subject:date:message-id:reply-to; bh=Tvm6bwW3l6an7jGgBlWvlKqEy/qLo6UEfVNuKIwgK3E=; b=CIPy1GFIL4VeFjOL5Y+WEJ0K5L4VdPXOcYYp/cIHSymQaaiR6/GXCgNTnVZqR/TL91 +sVSuq2fmdJpzUDtYPmfT2z7QUHtXvLqFlMGzAEjIpETJZNI8Aa13V+gSRsaAR+OzMH6 WfZod+T8tD7PJcPGicl8YJn2EByGDbmPdv013HklI0lEv4Q4oLXYKkKbCaDAWah8+SPn ahWrnMH1M0KN9eaIkXv6SOoIOXsddco/NMsofBnfA3K8VaRjo0DXeTIRCgQ0KHnKexls RO35NR5scDKxvBljgRrnui9TsoUDIg+Dv1ae3bscRl1rIju7hNgravK3A5UkDusKTlJU QSoQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=references:to:cc:in-reply-to:date:subject:mime-version:message-id :from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=Tvm6bwW3l6an7jGgBlWvlKqEy/qLo6UEfVNuKIwgK3E=; b=2fR5f0PCbxPKQPL9LUMPJczYW0sX1vIBRQ5WBVdO0Fw1QJ7mO4FF/NYIjFhxM4TvpX KFhMEnOiymelVww5X2bHLRkLB2xvb8R3fMzjxqy3yVh158gje146w/cBtNtiymR8kDXh jYL4zxhBx5TxyHBGun/E13tH/k9VuG0llaDZBHDgWOIY0cAK1H4TG3RQvlS53V/LfEbZ 1XqzHgZd+M0gvupfJ2Pxk3kXoLVPlgx2xU/HZT7epQnpYYyrJcG0RBwrYqq1eoQ5st+p CeLJpxPAqsEDxoNbyEzXz6EvqFFIMSHIlX99UWab3IzL+K3KXGo21yJYADhkq6QTFCsg CWtA== X-Gm-Message-State: ACrzQf30HVXv3pvRQwXX/3Cw/pgyir36JA1D9n1wYA2yf8RuL7fFqYq2 0L51oW3Z1N309Dphlij7TFb2YQY3NGUVSQ== X-Google-Smtp-Source: AMsMyM4y+Niz7LJ1xWHh+/Eqe3paIh4G55supwrW/xt5qr6FMInDtwySwpzHrfObFYhsKxt8KS3ktg== X-Received: by 2002:a05:6a00:4c89:b0:56e:4a14:89fa with SMTP id eb9-20020a056a004c8900b0056e4a1489famr23842714pfb.44.1667869883602; Mon, 07 Nov 2022 17:11:23 -0800 (PST) Received: from [172.17.252.129] (ns1.oxydns.net. [45.32.91.63]) by smtp.gmail.com with ESMTPSA id u2-20020a627902000000b0056b91044485sm5108339pfc.133.2022.11.07.17.11.22 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 07 Nov 2022 17:11:23 -0800 (PST) From: Zhenlei Huang Message-Id: <5CBAA944-5122-4BA0-854F-AF7D78ACF8AE@gmail.com> Content-Type: multipart/alternative; boundary="Apple-Mail=_56ADC306-43D0-4045-AA1D-8B26B5BC0396" List-Id: Networking and TCP/IP with FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-net List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-net@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.120.23.2.7\)) Subject: Re: GRE in a fib via rc.conf Date: Tue, 8 Nov 2022 09:11:18 +0800 In-Reply-To: Cc: freebsd-net@freebsd.org To: Mark Saad References: X-Mailer: Apple Mail (2.3608.120.23.2.7) X-Rspamd-Queue-Id: 4N5qmr6Ngjz3RK2 X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20210112 header.b=CIPy1GFI; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of zlei.huang@gmail.com designates 2607:f8b0:4864:20::531 as permitted sender) smtp.mailfrom=zlei.huang@gmail.com X-Spamd-Result: default: False [-1.50 / 15.00]; URI_COUNT_ODD(1.00)[15]; HTTP_TO_IP(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.998]; MV_CASE(0.50)[]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20210112]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-net@freebsd.org]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::531:from]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_COUNT_THREE(0.00)[3]; FREEMAIL_FROM(0.00)[gmail.com]; TO_DN_SOME(0.00)[]; TAGGED_FROM(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; FROM_EQ_ENVFROM(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RCVD_TLS_LAST(0.00)[]; MLMMJ_DEST(0.00)[freebsd-net@freebsd.org] X-ThisMailContainsUnwantedMimeParts: N --Apple-Mail=_56ADC306-43D0-4045-AA1D-8B26B5BC0396 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii > On Nov 8, 2022, at 8:26 AM, Mark Saad wrote: >=20 > All > I am looking for some help on if my setup makes sense.=20 > I have a vm with two interfaces. One for access to the host , we'll = call this mgmt . One for routing traffic, we'll call this routing . I = want to put the routing interface into a fib and to run a gre tunnel = over it. Sounds simple enough. The problem I am seeing is that it looks = like the tunneled traffic is leaked into the default fib and I don't see = why. I am not sure if this is config nit or if this is an issue. Should = the gre10 interface be in fib 1 ? See below.=20 The fib of the tunneling interface should also be 1 IIUC your setup. >=20 > ### RC CONF ### > ifconfig_vmx0=3D"inet 10.23.121.253/24 = description mgmt" > ifconfig_vmx1=3D"inet 100.65.101.14/28 mtu = 9000 description routing fib 1" > defaultrouter=3D"10.23.121.1" > static_routes=3D"ewr10gresrc" > route_ewr10gresrc=3D" 192.168.255.14 100.65.101.1 -fib 1" > cloned_interfaces=3D"gre10" > ifconfig_gre10=3D" inet 100.67.103.2 100.67.103.1 netmask = 255.255.255.252 tunnel 100.65.101.14 192.168.255.14 tunnelfib 1" Try this for the gre tunnel interface: cloned_interfaces=3D"gre10" create_args_gre10=3D"tunnel 100.65.101.14 192.168.255.14 tunnelfib 1" ifconfig_gre10=3D"inet 100.67.103.2 100.67.103.1 netmask 255.255.255.252 = fib 1" > ############### > =20 > ### DEFAULT FIB ### > ~ # netstat -nr4Wl > Routing tables > =20 > Internet: > Destination Gateway Flags Nhop# Mtu Netif Expire > default 10.23.121.1 UGS 6 1500 vmx0 > 10.23.121.0/24 link#1 U 2 1500 vmx0 > 10.23.121.253 link#1 UHS 3 16384 lo0 > 100.67.103.1 link#4 UH 4 1476 gre10 > 100.67.103.2 link#4 UHS 5 16384 lo0 > 127.0.0.1 link#3 UH 1 16384 lo0 > =20 > ### FIB 1 ### > =20 > # setfib 1 netstat -nr4Wl > Routing tables (fib: 1) > =20 > Internet: > Destination Gateway Flags Nhop# Mtu Netif Expire > 100.65.101.0/28 link#2 U 1 9000 vmx1 > 100.65.101.14 link#2 UHS 2 16384 lo0 > 127.0.0.1 link#3 UHS 3 16384 lo0 > 192.168.255.14 100.65.101.1 UGHS 4 9000 vmx1 > =20 > ##### PING EXAMPLES ##### > =20 > # setfib 1 ping -c 1 -t 2 100.67.103.1 > PING 100.67.103.1 (100.67.103.1): 56 data bytes > ping: sendto: No route to host > =20 > --- 100.67.103.1 ping statistics --- > 1 packets transmitted, 0 packets received, 100.0% packet loss > # setfib 0 ping -c 1 -t 2 100.67.103.1 > PING 100.67.103.1 (100.67.103.1): 56 data bytes > 64 bytes from 100.67.103.1 : icmp_seq=3D0 = ttl=3D255 time=3D1.528 ms > =20 > --- 100.67.103.1 ping statistics --- > 1 packets transmitted, 1 packets received, 0.0% packet loss > round-trip min/avg/max/stddev =3D 1.528/1.528/1.528/0.000 ms > =20 > #### TCPDUMP #### > ICMP packets are in fact sourced from the gre10 interface. > The GRE packets are also only going out the routing interface. >=20 > See the following pastebin for details. >=20 > https://pastebin.com/n3mGXGHA >=20 >=20 >=20 >=20 > --=20 > mark saad | nonesuch@longcount.org Best regards, Zhenlei --Apple-Mail=_56ADC306-43D0-4045-AA1D-8B26B5BC0396 Content-Transfer-Encoding: 7bit Content-Type: text/html; charset=us-ascii
On Nov 8, 2022, at 8:26 AM, Mark Saad <nonesuch@longcount.org> wrote:

All
  I am looking for some help on if my setup makes sense.
I have a vm with two interfaces. One for access to the host , we'll call this mgmt . One for routing traffic, we'll call this routing . I want to put the routing interface into a fib and to run a gre tunnel over it. Sounds simple enough.  The problem I am seeing is that it looks like the tunneled traffic is leaked into the default fib and I don't see why. I am not sure if this is config nit or if this is an issue. Should the gre10 interface be in fib 1 ?  See below.

The fib of the tunneling interface should also be 1 IIUC your setup.


### RC CONF ###
ifconfig_vmx0="inet 10.23.121.253/24 description mgmt"
ifconfig_vmx1="inet 100.65.101.14/28 mtu 9000 description routing fib 1"
defaultrouter="10.23.121.1"
static_routes="ewr10gresrc"
route_ewr10gresrc=" 192.168.255.14 100.65.101.1 -fib 1"
cloned_interfaces="gre10"
ifconfig_gre10=" inet 100.67.103.2 100.67.103.1 netmask 255.255.255.252 tunnel 100.65.101.14 192.168.255.14 tunnelfib 1"

Try this for the gre tunnel interface:

cloned_interfaces="gre10"
create_args_gre10="tunnel 100.65.101.14 192.168.255.14 tunnelfib 1"
ifconfig_gre10="inet 100.67.103.2 100.67.103.1 netmask 255.255.255.252 fib 1"

###############
 
### DEFAULT FIB ###
~ # netstat -nr4Wl
Routing tables
 
Internet:
Destination Gateway Flags Nhop# Mtu Netif Expire
default 10.23.121.1 UGS 6 1500 vmx0
10.23.121.0/24 link#1 U 2 1500 vmx0
10.23.121.253 link#1 UHS 3 16384 lo0
100.67.103.1 link#4 UH 4 1476 gre10
100.67.103.2 link#4 UHS 5 16384 lo0
127.0.0.1 link#3 UH 1 16384 lo0
 
### FIB 1 ###
 
# setfib 1 netstat -nr4Wl
Routing tables (fib: 1)
 
Internet:
Destination Gateway Flags Nhop# Mtu Netif Expire
100.65.101.0/28 link#2 U 1 9000 vmx1
100.65.101.14 link#2 UHS 2 16384 lo0
127.0.0.1 link#3 UHS 3 16384 lo0
192.168.255.14 100.65.101.1 UGHS 4 9000 vmx1
 
##### PING EXAMPLES #####
 
# setfib 1 ping -c 1 -t 2 100.67.103.1
PING 100.67.103.1 (100.67.103.1): 56 data bytes
ping: sendto: No route to host
 
--- 100.67.103.1 ping statistics ---
1 packets transmitted, 0 packets received, 100.0% packet loss
# setfib 0 ping -c 1 -t 2 100.67.103.1
PING 100.67.103.1 (100.67.103.1): 56 data bytes
64 bytes from 100.67.103.1: icmp_seq=0 ttl=255 time=1.528 ms
 
--- 100.67.103.1 ping statistics ---
1 packets transmitted, 1 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 1.528/1.528/1.528/0.000 ms
 
#### TCPDUMP ####
ICMP packets are in fact sourced from the gre10 interface.
The GRE packets are also only going out the routing interface.

See the following pastebin for details.





--
mark saad | nonesuch@longcount.org

Best regards,
Zhenlei

--Apple-Mail=_56ADC306-43D0-4045-AA1D-8B26B5BC0396--