IPSec transport mode, mtu, fragmentation...

[Eugene Grosbein]

18.01.2020 17:55, Victor Sudakov wrote:

>>>>> Back to the point. I've figured out that both encrypted (in transport
>>>>> mode) and unencrypted TCP segments have the same MSS=1460. Then I'm
>>>>> completely at a loss how the encrypted packets avoid being fragmented.
>>>>> TCP has no way to know in advance that encryption overhead will be
>>>>> added.
> 
> Here: http://admin.sibptus.ru/~vas/ftp-pcap.tar.gz you can find two
> identical FTP sessions, the only difference being ipsec=off during one
> session and ipsec=on during the other one.
> 
> As I said, in both the sessions MSS=1460 which is already odd, and I
> can't explain to myself why file transfer still works without MSS
> ajustment.
> 
> Moreover, something fishy is happening in the encrypted session: there
> are many TCP retransmissions (I was capturing on the FTP server's side,
> so there are many segments with the same sequence number). How would you
> explain this? There are almost no retransmissions in the unencrypted session.
> 
> All this is happening in a lab environment (one bhyve VM is an FTP
> server and the other downloads a file from the first), both VMs are on
> the same bridge interface. There are almost 19,000 packets in the
> encrypted file vs 12,000 in the plain file, I think because of those
> excessive retransmissions.
> 
> Could the retransmissions be some artifact of the enc(4) interface I was
> capturing the encrypted session on?

I doubt it. And I can't explain this, but maybe it's work of PMTUD Blackhole detection?
Look at sysctl net.inet.tcp | fgrep blackhole_