From nobody Tue Nov 08 01:34:57 2022 X-Original-To: freebsd-net@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4N5rJN1plFz4gyDy for ; Tue, 8 Nov 2022 01:35:16 +0000 (UTC) (envelope-from nonesuch@longcount.org) Received: from mail-il1-x133.google.com (mail-il1-x133.google.com [IPv6:2607:f8b0:4864:20::133]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4N5rJM1F51z3TL4 for ; Tue, 8 Nov 2022 01:35:15 +0000 (UTC) (envelope-from nonesuch@longcount.org) Received: by mail-il1-x133.google.com with SMTP id 7so6787777ilg.11 for ; Mon, 07 Nov 2022 17:35:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=longcount.org; s=google; h=to:subject:message-id:date:from:in-reply-to:references:mime-version :from:to:cc:subject:date:message-id:reply-to; bh=dGmxMhtIGA0daZSqn79tQu2LLJeMimYYpotQOa//+Iw=; b=mt90x4LfBjL544qXKVMRV8Ld0aduVJqKHytzAd6jBbEO7YfZj//ZNdJ66Z2B5mQAzD 6nD+Y2TIISHEFXuUsNmqffUAZwyNmuCMr6GtfSKZv1snimLuowr/Yt4UjX/BZS4Kn4Zr sTVfcP7Fxego6tSC4VpW6zj1Rp2VlxhBd+HiBwD9yrdIVRN/Vlu5bZz8I7GFQt+0+KWd vXZwOeu7PS9HoDT3ge9D8/vOERWNUOiY3o6tskevfUiXvYE6HI/qBbtnVyFxNf7DmtFL ozcLS4gJf2pKfIGNlfBace5HW9USNFgVtnkg3ABzreJMvrLwjSWage7knXxzb43AW/iK EL3w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=to:subject:message-id:date:from:in-reply-to:references:mime-version :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=dGmxMhtIGA0daZSqn79tQu2LLJeMimYYpotQOa//+Iw=; b=58SDGsVGNC6b5YDHLC6YirkSlw/pNNi6VUlBf6rJ8yaYwVYBPhqxcMPwI+FdOK25f2 /h3SwtQomT3XSNYCYdViuXekJ1XKwrMiXojIkxT1SOOG9KL4yQf5O7/I09SEw4Wa5RIZ HB6KBbdjtPUZQn9hiuUtUhJcxRtKCuonirwWJqXtCajMjKnxKDRqIUm4hRn8rGn2zvC5 0UhyCc0ow8b0WbLksjbz0wIwDHrQ77J7yAlOzpObDMhzsLoBl0S26wwWuf5u8SSDq07h EbWikn3fOiSVi+bjJr2ATh7mg0DIoO0L6xzQnZLj5/TTaIMfxC+Qa1I/MVVTjhC0tpI2 ACAA== X-Gm-Message-State: ACrzQf38WvpkZ1dI79WbQ4NL+ZSKiOv5LhOOTmhr1VCL0P5xo5G5/zeT NRKyChLZF72vFTx81dpc5QITooRfohAoTKN6lEJKm8cQNPe2QQ== X-Google-Smtp-Source: AMsMyM6LeA/Ge5MZraKnehheNVq4mkcPuz/WohC9nMSsGWMoAkPphzjOYxn1rJksQ+abPS1GRC59rKtAgcA5Rwi+hQU= X-Received: by 2002:a92:8708:0:b0:300:9123:586 with SMTP id m8-20020a928708000000b0030091230586mr29559316ild.156.1667871309474; Mon, 07 Nov 2022 17:35:09 -0800 (PST) List-Id: Networking and TCP/IP with FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-net List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-net@freebsd.org MIME-Version: 1.0 References: <5CBAA944-5122-4BA0-854F-AF7D78ACF8AE@gmail.com> In-Reply-To: <5CBAA944-5122-4BA0-854F-AF7D78ACF8AE@gmail.com> From: Mark Saad Date: Mon, 7 Nov 2022 20:34:57 -0500 Message-ID: Subject: Re: GRE in a fib via rc.conf To: freebsd-net@freebsd.org Content-Type: multipart/alternative; boundary="000000000000eec62505eceb8c6c" X-Rspamd-Queue-Id: 4N5rJM1F51z3TL4 X-Spamd-Bar: ++ Authentication-Results: mx1.freebsd.org; dkim=pass header.d=longcount.org header.s=google header.b=mt90x4Lf; dmarc=none; spf=pass (mx1.freebsd.org: domain of nonesuch@longcount.org designates 2607:f8b0:4864:20::133 as permitted sender) smtp.mailfrom=nonesuch@longcount.org X-Spamd-Result: default: False [2.33 / 15.00]; URI_COUNT_ODD(1.00)[9]; HTTP_TO_IP(1.00)[]; NEURAL_HAM_SHORT(-1.00)[-1.000]; NEURAL_SPAM_MEDIUM(0.98)[0.984]; NEURAL_SPAM_LONG(0.84)[0.844]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; R_DKIM_ALLOW(-0.20)[longcount.org:s=google]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; MLMMJ_DEST(0.00)[freebsd-net@freebsd.org]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::133:from]; RCVD_TLS_LAST(0.00)[]; ARC_NA(0.00)[]; DMARC_NA(0.00)[longcount.org]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; DKIM_TRACE(0.00)[longcount.org:+]; PREVIOUSLY_DELIVERED(0.00)[freebsd-net@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; TO_DN_NONE(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-ThisMailContainsUnwantedMimeParts: N --000000000000eec62505eceb8c6c Content-Type: text/plain; charset="UTF-8" On Mon, Nov 7, 2022 at 8:11 PM Zhenlei Huang wrote: > > On Nov 8, 2022, at 8:26 AM, Mark Saad wrote: > > All > I am looking for some help on if my setup makes sense. > I have a vm with two interfaces. One for access to the host , we'll call > this mgmt . One for routing traffic, we'll call this routing . I want to > put the routing interface into a fib and to run a gre tunnel over it. > Sounds simple enough. The problem I am seeing is that it looks like the > tunneled traffic is leaked into the default fib and I don't see why. I am > not sure if this is config nit or if this is an issue. Should the gre10 > interface be in fib 1 ? See below. > > > The fib of the tunneling interface should also be 1 IIUC your setup. > > > ### RC CONF ### > ifconfig_vmx0="inet 10.23.121.253/24 description mgmt" > ifconfig_vmx1="inet 100.65.101.14/28 mtu 9000 description routing fib 1" > defaultrouter="10.23.121.1" > static_routes="ewr10gresrc" > route_ewr10gresrc=" 192.168.255.14 100.65.101.1 -fib 1" > cloned_interfaces="gre10" > ifconfig_gre10=" inet 100.67.103.2 100.67.103.1 netmask 255.255.255.252 > tunnel 100.65.101.14 192.168.255.14 tunnelfib 1" > > > Try this for the gre tunnel interface: > > cloned_interfaces="gre10" > create_args_gre10="tunnel 100.65.101.14 192.168.255.14 tunnelfib 1" > ifconfig_gre10="inet 100.67.103.2 100.67.103.1 netmask 255.255.255.252 fib > 1" > > Good catch , and I confirmed it works in either format ifconfig_gre10=" inet 100.67.103.2 100.67.103.1 netmask 255.255.255.252 fib 1 tunnel 100.65.101.14 192.168.255.14 tunnelfib 1" or create_args_gre10="tunnel 100.65.101.14 192.168.255.14 tunnelfib 1" ifconfig_gre10="inet 100.67.103.2 100.67.103.1 netmask 255.255.255.252 fib 1" I don't think this is documented very well. Thanks for your help. ############### > > ### DEFAULT FIB ### > ~ # netstat -nr4Wl > Routing tables > > Internet: > Destination Gateway Flags Nhop# Mtu Netif Expire > default 10.23.121.1 UGS 6 1500 vmx0 > 10.23.121.0/24 link#1 U 2 1500 vmx0 > 10.23.121.253 link#1 UHS 3 16384 lo0 > 100.67.103.1 link#4 UH 4 1476 gre10 > 100.67.103.2 link#4 UHS 5 16384 lo0 > 127.0.0.1 link#3 UH 1 16384 lo0 > > ### FIB 1 ### > > # setfib 1 netstat -nr4Wl > Routing tables (fib: 1) > > Internet: > Destination Gateway Flags Nhop# Mtu Netif Expire > 100.65.101.0/28 link#2 U 1 9000 vmx1 > 100.65.101.14 link#2 UHS 2 16384 lo0 > 127.0.0.1 link#3 UHS 3 16384 lo0 > 192.168.255.14 100.65.101.1 UGHS 4 9000 vmx1 > > ##### PING EXAMPLES ##### > > # setfib 1 ping -c 1 -t 2 100.67.103.1 > PING 100.67.103.1 (100.67.103.1): 56 data bytes > ping: sendto: No route to host > > --- 100.67.103.1 ping statistics --- > 1 packets transmitted, 0 packets received, 100.0% packet loss > # setfib 0 ping -c 1 -t 2 100.67.103.1 > PING 100.67.103.1 (100.67.103.1): 56 data bytes > 64 bytes from 100.67.103.1: icmp_seq=0 ttl=255 time=1.528 ms > > --- 100.67.103.1 ping statistics --- > 1 packets transmitted, 1 packets received, 0.0% packet loss > round-trip min/avg/max/stddev = 1.528/1.528/1.528/0.000 ms > > #### TCPDUMP #### > ICMP packets are in fact sourced from the gre10 interface. > The GRE packets are also only going out the routing interface. > > See the following pastebin for details. > > https://pastebin.com/n3mGXGHA > > > > > -- > mark saad | nonesuch@longcount.org > > > Best regards, > Zhenlei > > -- mark saad | nonesuch@longcount.org --000000000000eec62505eceb8c6c Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


=
On Mon, Nov 7, 2022 at 8:11 PM Zhenle= i Huang <zlei.huang@gmail.com> wrote:

<= br>
All
=C2=A0 I am looking for some he= lp on if my setup makes sense.
I have a vm with two interfaces. One for access to the host , we'll call= =20 this mgmt . One for routing traffic, we'll call this routing . I want t= o put the routing interface into a fib and to run a gre tunnel over it.=20 Sounds simple enough.=C2=A0 The problem I am seeing is that it looks like t= he tunneled traffic is leaked into the default fib and I don't see why. I= =20 am not sure if this is config nit or if this is an issue. Should the=20 gre10 interface be in fib 1 ?=C2=A0 See below.

The fib of the tunneling interface should also b= e 1 IIUC your setup.


### RC CONF ###
ifconfig_vmx0=3D"inet 10.23.121.253/24 description mgmt"
ifconfig_vmx1=3D"inet 100.65.101.14/28 mtu 9000 description routing fib 1&qu= ot;
defaultrouter=3D"10.23.121.1"
static_routes=3D"ewr10gresrc"
route_ewr10gresrc=3D" 192.168.255.14 100.65.101.1 -fib 1&qu= ot;
cloned_interfaces=3D"gre10"
ifconfig_gre10=3D" inet 100.67.103.2 100.67.103.1 netmask 2= 55.255.255.252 tunnel 100.65.101.14 192.168.255.14 tunnelfib 1"

Try this for the g= re tunnel interface:

cloned_interfaces=3D"gre= 10"
create_args_gre10=3D"tunnel 100.65.101.14 192.168.2= 55.14 tunnelfib 1"
ifconfig_gre10=3D"inet 100.67.103.2 = 100.67.103.1 netmask 255.255.255.252 fib 1"


Good catch , and I confirmed it works in either= format

ifconfig_gre10=3D" inet 100.67.1= 03.2 100.67.103.1 netmask 255.255.255.252 fib 1 tunnel 100.65.101.14 192.16= 8.255.14 tunnelfib 1"
=C2=A0or
create_args_gre10=3D"tunnel 100.65.= 101.14 192.168.255.14 tunnelfib 1"
ifconfig_gre10=3D"in= et 100.67.103.2 100.67.103.1 netmask 255.255.255.252 fib 1"
=
I don't think this is documented very well. Thanks for y= our help.

###############
=C2=A0
### DEFAULT FIB ###
~ # netstat -nr4Wl
Routing tables
=C2=A0
Internet:
Destination Gateway Flags Nhop# Mtu = Netif Expire
default 10.23.121.1 UGS 6 1500 = vmx0
10.23.121.0/= 24 link#1 U 2 1500 vmx0
10.23.121.253 link#1 UHS 3 16384 = lo0
100.67.103.1 link#4 UH 4 1476 = gre10
100.67.103.2 link#4 UHS 5 16384 = lo0
127.0.0.1 link#3 UH 1 16384 = lo0
=C2=A0
### FIB 1 ###
=C2=A0
# setfib 1 netstat -nr4Wl
Routing tables (fib: 1)
=C2=A0
Internet:
Destination Gateway Flags Nhop# Mtu = Netif Expire
100.65.101.= 0/28 link#2 U 1 9000 vmx1
100.65.101.14 link#2 UHS 2 16384 = lo0
127.0.0.1 link#3 UHS 3 16384 = lo0
192.168.255.14 100.65.101.1 UGHS 4 9000 = vmx1
=C2=A0
##### PING EXAMPLES #####
=C2=A0
# setfib 1 ping -c 1 -t 2 100.67.103.1
PING 100.67.103.1 (100.67.103.1): 56 data bytes
ping: sendto: No route to host
=C2=A0
--- 100.67.103.1 ping statistics ---
1 packets transmitted, 0 packets received, 100.0% packet loss
# setfib 0 ping -c 1 -t 2 100.67.103.1
PING 100.67.103.1 (100.67.103.1): 56 data bytes
64 bytes from 100.67.103.1: icmp_seq=3D0 ttl=3D255 time=3D1.528 ms
=C2=A0
--- 100.67.103.1 ping statistics ---
1 packets transmitted, 1 packets received, 0.0% packet loss
round-trip min/avg/max/stddev =3D 1.528/1.528/1.528/0.000 ms
=C2=A0
#### TCPDUMP ####
ICMP packets are in fact sourced from the gre10 interface.=20
The GRE packets are also only going out the routing interface.
<= div>
See the following pastebin for details.

Best rega= rds,
Zhenlei



--
mark saad | nonesuch@longcount.org
--000000000000eec62505eceb8c6c--