From nobody Thu Dec 15 22:03:32 2022 X-Original-To: freebsd-cloud@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4NY5pn069jz4dLyd for ; Thu, 15 Dec 2022 22:03:45 +0000 (UTC) (envelope-from robballantyne3@gmail.com) Received: from mail-vs1-xe31.google.com (mail-vs1-xe31.google.com [IPv6:2607:f8b0:4864:20::e31]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4NY5pm0cfHz4P71 for ; Thu, 15 Dec 2022 22:03:44 +0000 (UTC) (envelope-from robballantyne3@gmail.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20210112 header.b=KOw8o25e; spf=pass (mx1.freebsd.org: domain of robballantyne3@gmail.com designates 2607:f8b0:4864:20::e31 as permitted sender) smtp.mailfrom=robballantyne3@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-vs1-xe31.google.com with SMTP id h26so613077vsr.5 for ; Thu, 15 Dec 2022 14:03:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=to:subject:message-id:date:from:mime-version:from:to:cc:subject :date:message-id:reply-to; bh=8sP0x8eaP1jPeehchKh8X+So3ro+p361K7i7/fA4wy0=; b=KOw8o25elrX09JWDLQrEMfr6/CKDpnX/0KEf8PKIsy+675ssBoaU7U7+qHuV2EjZe8 fhG/fpmyc4RNaarYMdNM8YUgWv3UHHF0vZAPdWENDA2vaThvKdSN+dPoLGy09aUNXMYU HKFwbatv7CwnFw7HbyH392WUWS4+N9O9RT21FjCku6wpRbEzWJtsKGVvaynG+umuPAuH AgXJ7eg8T5rHMP2qjM2Pg16Hwkqa5x9o0P2PyiY/RJteEwbgQGIoakRqJmZZeI+nq4kk V7D3w3BJbQSQHZGvP58vJYWatvslxX6uWXmEnwZRkXHpqy7rzxOgeFPn8/iGCXQTQ3bu 6gcg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=to:subject:message-id:date:from:mime-version:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=8sP0x8eaP1jPeehchKh8X+So3ro+p361K7i7/fA4wy0=; b=KuZliLRD8k2ikWvxzbG8uw6s3PyU70+pRgN+7wZhxbwxFYueVX95zrE5yVneIZqh+d 5f1DndR90dxheEoH6R35vLgpLl1Wsg3e/FhEpJ1jhBADTUmK0mN8gEKgdUjlBhLsFxf4 53Ow3b47Cm/sHGQ6IqaZWfKgRTUhRvHK+FfxGKaS5dXQRAGi32c50fQtY5epJyrsA5Qk 4awGe9w7kcLqHZN2SDp62nfjgfv+7CmRffK7m6sxKNmNRdF32M5VX5Tzz3y+uEIcYQtD RXVbo0AcW2XzCuyr6aPBbEnfBqqulRKeO+5snnRr3HO5x5fedfiBIT6BOIc3SUqc0zz4 ozbA== X-Gm-Message-State: ANoB5pnC5icxyzlUAEvzR3uXBbRUSqlsiqcp9nG8LNGu5hVyfnu0CF+f 32ngAPwmiFn3mS1G4wgJYPJcw5UmUBsMOy5RkwGDz7u78IY= X-Google-Smtp-Source: AA0mqf4xKqbsUaoUCshmKIQDAOyejdsnYTmoNXpTvC56Q3sZPahuh+m5P2VWKm87QNB6QleOCFfflHp4h20Awow/ejc= X-Received: by 2002:a67:1744:0:b0:3b0:8e7c:5f72 with SMTP id 65-20020a671744000000b003b08e7c5f72mr35654231vsx.27.1671141823423; Thu, 15 Dec 2022 14:03:43 -0800 (PST) List-Id: FreeBSD on cloud platforms (EC2, GCE, Azure, etc.) List-Archive: https://lists.freebsd.org/archives/freebsd-cloud List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-cloud@freebsd.org MIME-Version: 1.0 From: Rob Ballantyne Date: Thu, 15 Dec 2022 14:03:32 -0800 Message-ID: Subject: What is a VPC (google's specifically but it could be more general) really? To: freebsd-cloud@freebsd.org Content-Type: multipart/alternative; boundary="000000000000c128d905efe506ec" X-Spamd-Result: default: False [-1.97 / 15.00]; SUBJECT_ENDS_QUESTION(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; HTTP_TO_IP(1.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.97)[-0.974]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20210112]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36:c]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; MIME_TRACE(0.00)[0:+,1:+,2:~]; PREVIOUSLY_DELIVERED(0.00)[freebsd-cloud@freebsd.org]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::e31:from]; FREEMAIL_FROM(0.00)[gmail.com]; ARC_NA(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; DKIM_TRACE(0.00)[gmail.com:+]; MID_RHS_MATCH_FROMTLD(0.00)[]; TO_DN_NONE(0.00)[]; MLMMJ_DEST(0.00)[freebsd-cloud@freebsd.org]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_COUNT_TWO(0.00)[2] X-Rspamd-Queue-Id: 4NY5pm0cfHz4P71 X-Spamd-Bar: - X-ThisMailContainsUnwantedMimeParts: N --000000000000c128d905efe506ec Content-Type: text/plain; charset="UTF-8" Hello, I have a question about what the internal structure and forwarding is within Google's VPCs. I started into a project using OpenVPN to bind my home network to an isolated VPC in Google's Cloud when I discovered the routing didn't work quite the way I thought. I had assumed that VPCs would look like a private VLAN (Layer2) into which Google's infrastructure would inject L3 router interfaces and/or ip/ethernet filters. I set up a private VPC and two test FreeBSD boxes to test and see exactly how VPC configures routing. First, I just used a standard install of 13.1 and the routing table after everything is up and configured looks like: ---- Internet: Destination Gateway Flags Netif Expire default 10.1.1.1 UGS vtnet0 10.1.1.1 link#1 UHS vtnet0 10.1.1.20 link#1 UH lo0 127.0.0.1 link#2 UH lo0 ---- This looked a little unusual to me so (there was no link local route for all the addresses in the VPC), I commented out the rc.conf entry 'google_network_daemon_enable=YES' and setup the vtnet0 interface up manually with: 'ifconfig_vtnet0="inet 10.1.1.20 netmask 255.255.255.0"' The resulting routing table: ---- Internet: Destination Gateway Flags Netif Expire 10.1.1.0/24 link#1 U vtnet0 10.1.1.20 link#1 UHS lo0 127.0.0.1 link#2 UH lo0 ---- This configuration wasn't able to communicate. The latter routing table looks more usual though, with a 10.1.1.0/24 route to the local link. So, it appears to me that VPCs are really configured to be a point-to-point (star really) network where the Google router interface (10.1.1.1 in this case) has to handle all forwarding between nodes of a network. I've searched around the web to try and confirm this but there is scant detail on how exactly forwarding works within a single VPC. My VPN project involved using a bastion VPN host that would have terminated the VPN/SSL tunnel and routed traffic between my home network and the isolated network behind the bastion. Before I make final decisions on configuration, I wanted to know if my understanding is correct and whether there is any documentation on this that I've somehow missed. FreeBSD is, of course, the host of choice for this operation! If anyone does know any details, any info would be greatly appreciated. Many Thanks, Rob Ballantyne --000000000000c128d905efe506ec Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hello,

=C2=A0 I have a question about what the internal= structure and forwarding is within Google's VPCs.

=C2=A0 I star= ted into a project using OpenVPN to bind my home network to an isolated VPC= in Google's Cloud when I discovered the routing didn't work quite = the way I thought.=C2=A0 I had assumed that VPCs would look like a private = VLAN (Layer2) into which Google's infrastructure would inject L3 router= interfaces and/or ip/ethernet filters.

=C2=A0 I set up a private VP= C and two test FreeBSD boxes to test and see exactly how VPC configures rou= ting.=C2=A0=C2=A0

=C2=A0 First, I just used a standard install of 13= .1 and the routing table after everything is up and configured looks like:<= /div>

----
Internet:
Destination =C2=A0 =C2=A0 =C2=A0 =C2=A0Gateway = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Flags =C2=A0 =C2=A0 Netif Expiredefault =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A010.1.1.1 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 UGS =C2=A0 =C2=A0 =C2=A0vtnet0
10.1.1.1 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 link#1 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 U= HS =C2=A0 =C2=A0 =C2=A0vtnet0
10.1.1.20 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0link#1 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 UH =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0lo0
127.0.0.1 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0link#2 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 UH =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0lo0
----

=C2=A0 This looked a little unusual to me so (there= was no link local route for all the addresses in the VPC), I commented out= the rc.conf entry 'google_network_daemon_enable=3DYES' and setup t= he vtnet0 interface up manually with: 'ifconfig_vtnet0=3D"inet 10.= 1.1.20 netmask 255.255.255.0"'=C2=A0 The resulting routing table:<= /div>

----
Internet:
Destination =C2=A0 =C2=A0 =C2=A0 =C2=A0Gateway = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Flags =C2=A0 =C2=A0 Netif Expire10.1.1.0/24 =C2=A0 =C2=A0 =C2=A0 =C2= =A0link#1 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 U =C2=A0 =C2=A0 =C2=A0 = =C2=A0vtnet0
10.1.1.20 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0link#1 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 UHS =C2=A0 =C2=A0 =C2=A0 =C2=A0 lo0
1= 27.0.0.1 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0link#2 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 UH =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0lo0
----
<= div class=3D"gmail_default" style=3D"font-family:monospace,monospace">
<= /div>
=C2=A0 This configuration wasn't able to communicate. The latter routi= ng table looks more usual though, with a 10.= 1.1.0/24 route to the local link.

=C2=A0 So, it appears to me th= at VPCs=C2=A0are really configured to be a point-to-point (star really) net= work where the Google router interface (10.1.1.1 in this case) has to handl= e all forwarding between nodes of a network.

=C2=A0 I've searche= d around the web to try and confirm this but there is scant detail on how e= xactly forwarding works within a single VPC.

=C2=A0 My VPN project i= nvolved using a bastion VPN host that would have terminated the VPN/SSL tun= nel and routed traffic between my home network and the isolated network beh= ind the bastion.

=C2=A0 Before I make final decisions on configurati= on, I wanted=C2=A0to know if my understanding is correct and whether there = is any documentation on this that I've somehow missed.

=C2=A0 F= reeBSD is, of course, the host of choice for this operation!

=C2=A0 = If anyone does know any details, any info would be greatly appreciated.
Many Thanks,
Rob Ballantyne
--000000000000c128d905efe506ec--