IPSec transport mode, mtu, fragmentation...

[Eugene Grosbein]

17.01.2020 22:09, Victor Sudakov wrote:

>>> Back to the point. I've figured out that both encrypted (in transport
>>> mode) and unencrypted TCP segments have the same MSS=1460. Then I'm
>>> completely at a loss how the encrypted packets avoid being fragmented.
>>> TCP has no way to know in advance that encryption overhead will be
>>> added.
>>
>> If outgoing route (f.e. default route) has lower MTU, kernel should respond with EMSGSIZE
>> to TCP's attempt to send oversized packet when PMTUD is enabled.
>>
>> If PMTUD discovers that path mtu is low, it should store this information in the hostcache
>> (see sysctl net.inet.tcp.hostcache.list) and use hostcache's MTU for same goal.
> 
> Should this result in a smaller MSS in TCP to such hosts?
> 
> PS "sysctl net.inet.tcp.hostcache.list | grep 192.168.246.11" yields
> nothing, and yet 192.168.246.11 is the VM with which I have a transport
> mode SA.

I guess that Path MTU in such case is 1500, so PMTUD won't help here.