Re: What is a VPC (google's specifically but it could be more general) really?

Hi,

I don't have a direct answer, but as a user I can confirm that OpenVPN in
layer 3 mode works for me. I simply haven't tried it in layer 2 mode with
GCE (because I've no need for layer 2 and it incurs the extra overhead of
broadcast packets). Layer 2 mode probably won't work anyway because the MTU
has to be reduced to 1460, unless you do that on all participating hosts.
Point is, if that's an option for you it might be worth exploring.

As a side note, I configure the tun devices with the same IP address at the
vtnet device. That actually works perfectly, even though the two endpoints
are on wildly different networks, and avoids maintaining DNS entries and
routes for the point to point network.

For example:
vtnet0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0
mtu 1460
    inet 10.1.2.3 netmask 0xffffffff broadcast 10.1.2.3
    inet 130.x.x.x netmask 0xffffffff broadcast 130.x.x.x
tun5000: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> metric 0 mtu 1500
    inet 10.1.2.3 --> 10.16.0.1 netmask 0xffffffff

(The internal IP on my home router's ethernet interface is 10.16.0.1/16,
overlapping with its tun interface.)

-Leif





On Thu, Dec 15, 2022 at 4:03 PM Rob Ballantyne <robballantyne3@gmail.com>
wrote:

> Hello,
>
>   I have a question about what the internal structure and forwarding is
> within Google's VPCs.
>
>   I started into a project using OpenVPN to bind my home network to an
> isolated VPC in Google's Cloud when I discovered the routing didn't work
> quite the way I thought.  I had assumed that VPCs would look like a private
> VLAN (Layer2) into which Google's infrastructure would inject L3 router
> interfaces and/or ip/ethernet filters.
>
>   I set up a private VPC and two test FreeBSD boxes to test and see
> exactly how VPC configures routing.
>
>   First, I just used a standard install of 13.1 and the routing table
> after everything is up and configured looks like:
>
> ----
> Internet:
> Destination        Gateway            Flags     Netif Expire
> default            10.1.1.1           UGS      vtnet0
> 10.1.1.1           link#1             UHS      vtnet0
> 10.1.1.20          link#1             UH          lo0
> 127.0.0.1          link#2             UH          lo0
> ----
>
>   This looked a little unusual to me so (there was no link local route for
> all the addresses in the VPC), I commented out the rc.conf entry
> 'google_network_daemon_enable=YES' and setup the vtnet0 interface up
> manually with: 'ifconfig_vtnet0="inet 10.1.1.20 netmask 255.255.255.0"'
> The resulting routing table:
>
> ----
> Internet:
> Destination        Gateway            Flags     Netif Expire
> 10.1.1.0/24        link#1             U        vtnet0
> 10.1.1.20          link#1             UHS         lo0
> 127.0.0.1          link#2             UH          lo0
> ----
>
>   This configuration wasn't able to communicate. The latter routing table
> looks more usual though, with a 10.1.1.0/24 route to the local link.
>
>   So, it appears to me that VPCs are really configured to be a
> point-to-point (star really) network where the Google router interface
> (10.1.1.1 in this case) has to handle all forwarding between nodes of a
> network.
>
>   I've searched around the web to try and confirm this but there is scant
> detail on how exactly forwarding works within a single VPC.
>
>   My VPN project involved using a bastion VPN host that would have
> terminated the VPN/SSL tunnel and routed traffic between my home network
> and the isolated network behind the bastion.
>
>   Before I make final decisions on configuration, I wanted to know if my
> understanding is correct and whether there is any documentation on this
> that I've somehow missed.
>
>   FreeBSD is, of course, the host of choice for this operation!
>
>   If anyone does know any details, any info would be greatly appreciated.
>
> Many Thanks,
> Rob Ballantyne
>