Packet forwarding stooped when Strongswan install IPsec policy

From: Victor Gamov <vitspec_at_gmail.com>
Date: Sat, 14 Oct 2023 10:25:23 UTC
Hi All

I have FreeBSD 13.2-STABLE stable/13-n255939-b9da47180fd6 GENERIC amd64
machine with strongswan-5.9.11_2 installed by pkg.

When routed ipsec is up all outgoing packets forwarded into ipsec-tunnel so
networking is immediately fails.

FreeBSD config:
=====
net.fibs=4
net.inet.ip.forwarding=1
=====


ifconfig ipsec10121
=====
ipsec10121: flags=8050<UP,POINTOPOINT,RUNNING,MULTICAST> metric 0 mtu 1400
description: PoP-12
tunnel inet 1.1.1.2 --> 2.2.2.2
inet 172.16.110.129 --> 172.16.110.130 netmask 0xfffffffc
groups: ipsec
reqid: 10121
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
=====


strongswan etc/ipsec.conf:
=====
conn pop4-to-pop12-routed
#  also = tmpl_route_based
  left = 1.1.1.2
  right = 2.2.2.2
  leftsubnet = 0.0.0.0/0
  rightsubnet = 0.0.0.0/0
  reqid = 10121
  type = tunnel
  authby = psk
  keyexchange = ikev2
  ike = aes256-sha256-modp3072,aes256-sha256-modp3072
  esp = aes256-sha256-modp3072,aes256-sha256-modp3072
  ikelifetime = 28800
  mobike = no
  lifetime = 3600
  dpdaction = restart
  dpddelay = 30s
  auto = start
=====


strongswan etc/strongswan.d/charon/kernel-pfkey.conf:
=====
kernel-pfkey {
  load = yes
# route_via_internal = no
}
=====


route -n monitor
=====
got message of size 272 on Sat Oct 14 12:39:39 2023
RTM_GET: Report Metrics: len 272, pid: 49695, seq 1, errno 0,
flags:<UP,GATEWAY,DONE,STATIC>
locks:  inits:
sockaddrs: <DST,GATEWAY,NETMASK,IFP,IFA>
 0.0.0.0 1.1.1.1 0.0.0.0 vlan200:48.dc.2d.6.4f.f4 1.1.1.2

got message of size 200 on Sat Oct 14 12:39:39 2023
RTM_GET: Report Metrics: len 200, pid: 49695, seq 2, errno 0,
flags:<UP,GATEWAY,DONE,STATIC>
locks:  inits:
sockaddrs: <DST,GATEWAY,NETMASK>
 0.0.0.0 1.1.1.1 0.0.0.0

got message of size 256 on Sat Oct 14 12:39:39 2023
RTM_ADD: Add Route: len 256, pid: 49695, seq 3, errno 0,
flags:<UP,GATEWAY,HOST,DONE,STATIC>
locks:  inits:
sockaddrs: <DST,GATEWAY,IFP,IFA>
 2.2.2.2 1.1.1.1 vlan200:48.dc.2d.6.4f.f4 1.1.1.2

got message of size 272 on Sat Oct 14 12:39:39 2023
RTM_ADD: Add Route: len 272, pid: 49695, seq 5, errno 0,
flags:<UP,DONE,STATIC>
locks:  inits:
sockaddrs: <DST,GATEWAY,NETMASK,IFP,IFA>
 128.0.0.0 1.1.1.1 128.0.0.0 vlan200:48.dc.2d.6.4f.f4 1.1.1.2

got message of size 272 on Sat Oct 14 12:39:39 2023
RTM_ADD: Add Route: len 272, pid: 49695, seq 4, errno 0,
flags:<UP,DONE,STATIC>
locks:  inits:
sockaddrs: <DST,GATEWAY,NETMASK,IFP,IFA>
 0.0.0.0 1.1.1.1 128.0.0.0 vlan200:48.dc.2d.6.4f.f4 1.1.1.2
=====


netstat -r -nW4:
=====
Routing tables

Internet:
Destination        Gateway            Flags   Nhop#    Mtu      Netif Expire
0.0.0.0/1          195.34.58.166      US         12   1500    vlan200
default            195.34.58.166      UGS         6   1500    vlan200
10.4.102.128/31    link#8             U           8   1500     vlan22
10.4.102.129       link#8             UHS         7  16384        lo0
31.131.95.64/27    127.0.0.1          U1B         9  16384        lo0
46.243.226.103     195.34.58.166      UGHS       10   1500    vlan200
127.0.0.1          link#5             UHS         1  16384        lo0
128.0.0.0/1        195.34.58.166      US         12   1500    vlan200
172.16.110.12/31   link#4             U           2   1500       ixl3
172.16.110.13      link#4             UHS         3  16384        lo0
172.16.110.129     link#11            UHS        11  16384        lo0
195.34.58.166/31   link#7             U           4   1500    vlan200
195.34.58.167      link#7             UHS         5  16384        lo0
=====


netstat -o -nW4
=====
Nexthop data

Internet:
Idx   Type         IFA                Gateway             Flags      Use
Mtu         Netif     Addrif Refcnt Prepend
1       v4/resolve 127.0.0.1          lo0/resolve        HS         1366
 16384        lo0               2
2       v4/resolve 172.16.110.13      ixl3/resolve                     0
1500       ixl3               2
3       v4/resolve 127.0.0.1          lo0/resolve        HS            0
 16384        lo0      ixl3     2
4       v4/resolve 195.34.58.167      vlan200/resolve              51749
1500    vlan200               4
5       v4/resolve 127.0.0.1          lo0/resolve        HS            0
 16384        lo0   vlan200     2
6            v4/gw 195.34.58.167      195.34.58.166      GS        37902
1500    vlan200               2
7       v4/resolve 127.0.0.1          lo0/resolve        HS            0
 16384        lo0    vlan22     2
8       v4/resolve 10.4.102.129       vlan22/resolve                   3
1500     vlan22               2
9       v4/resolve 127.0.0.1          lo0/resolve        1B            0
 16384        lo0               2
10           v4/gw 195.34.58.167      195.34.58.166      GHS           0
1500    vlan200               2
11      v4/resolve 127.0.0.1          lo0/resolve        HS            0
 16384        lo0ipsec10121     2
12      v4/resolve 195.34.58.167      vlan200/resolve    S             0
1500    vlan200               3
=====


If I changed "route_via_internal=yes" at
etc/strongswan.d/charon/kernel-pfkey.conf then no route like 0.0.0.0/1 or
128.0.0.0/1 installed but network still fails

The very same strongswan config works fine for many years on FreeBSD-11.
FreeBSD-13 has many changes at network stack and strongswan changed too.

Also I read  https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=255678  and
https://github.com/strongswan/strongswan/issues/910 and its looks like
strongswan/FreeBSD integration issue.


I'll appreciate any advice.  Thanks!

-- 
CU,
Victor Gamov