Software IPSEC AES-CTR/SHA-256 broken

[Eugene Grosbein]

Hi!

I used IPSEC/gif (ipencap) with static Blowfish/SHA1 keys for years for VPN tunnels between FreeBSD routers.

After recent Blowfish deprecation notice, I tried switching to AES-CTR/SHA-256 instead
only to find that transit RDP TCP streams start to break often (every 20 minutes or so).
I switched back to Blowfish/SHA1 and problem disappeared.

While using AES-CTR/SHA-256, "netstat -p esp -ss" shows increasing counters for:

 packets dropped; bad encryption detected
 packets dropped; bad authentication detected

These counters stay zero when only Blowfish is used.
Both sides have no AES-NI hardware support and aesni.ko kernel module is not even loaded.
FreeBSD 11.2-STABLE/amd64 r343942 at one side is used and
11.2-STABLE/amd64 r343953 at another side.

I managed to reproduce the problem using simple ssh connection over tunnel built with commands:

ifconfig gif1 create tunnel 1.1.1.1 2.2.2.2
ifconfig gif1 inet 192.168.80.242 192.168.80.241 netmask 255.255.255.252 mtu 1440

And /etc/ipsec.conf:

flush;
spdflush;
add 1.1.1.1 2.2.2.2 esp 1013 -m transport -E aes-ctr "M28_)KDFV,iFVHNIKO-p" -A hmac-sha2-256 "rdijokg&YRDYi(I%R0oMJN%()H532d92";
add 2.2.2.2 1.1.1.1 esp 2013 -m transport -E aes-ctr "G5of)Gbn556reg_+;mVz" -A hmac-sha2-256 "?op;_rf{89CE$DC09*3$RFgi7y9)i-e]";

spdadd 2.2.2.2/32 1.1.1.1/32 icmp -P out none;
spdadd 2.2.2.2/32 1.1.1.1/32 esp -P out none;
spdadd 2.2.2.2/32 1.1.1.1/32 any -P out ipsec
  esp/transport//require;
spdadd 1.1.1.1/32 2.2.2.2/32 any -P out ipsec
  esp/transport//use;
#EOF

No IKE daemon used. Another side has symmetric settings. The problem reproduces while
making ssh connection from 192.168.80.242 to 192.168.80.241:

$ ssh -vp 10022 192.168.80.241 dd if=random.bin > /dev/null

The file random.bin contains some data obtained from /dev/urandom.

N.B.: SSH connection does not break as it tolerates delays produced by TCP retransmits
much better comparing to RDP.