[IPsec] Weird performance issue via IPsec/racoon tunnel

[Michael Grimm]

Eugene Grosbein <eugen at grosbein.net> wrote:
> 10.12.2017 23:55, Michael Grimm wrote:


> "bad cksum 0" is pretty normal for traffic going out via interface supporting hardware checksum offload,
> so kernel skips computing checksum before passing packets to the NIC.

Ok, good to know.

> Your problem more likely is due to fragmented ESP packets.
> It's not uncommon when cloud IP stack or ISP infrastructure drop high percentage
> of fragmented ESP packets because they are not optimized for such packets,
> e.g. router has to process them in software instead of hardware
> like non-fragmented packets are processed.

Thank you for this explanation. 

I did already lower MTU: If I do configure vtnet0 to a MTU of 1490 at boot time I do not not notice a performance loss compared to the default 1500 setting.

>> *BUT* if I do a "ifconfig vtnet0 mtu 1450 up ; ifconfig vtnet0 mtu 1500 up" I do observe:
>> 
>> 	#) scp NEW to OLD via IPsec tunnel:	17.1 MB/s !
>> 	#) scp OLD to NEW via IPsec tunnel:	16.9 MB/s


*BUT* if I do boot with the default 1500 setting, changing the MTU to e.g. 1450 and *immediately* back to 1500 manually, I do not encounter any performance loss at all. Why? Even when booting 1490 and immediately setting the MTU manually to 1500 I do not see any performance loss. Strange.

> When you lower MTU of vtnet enough to make encapsulated packets (payload+overhead) <=1500 bytes,
> resulted ESP packets have not be fragmented and pass just fine.

I will keep the MTU at 1490 and monitor that server for the time being.

> To verify if it's your case, you should run two tcpdump commands,
> one at sending side and another at receiving size 
> and compare outputs to see if *every* outgoing packet reaches its destination or not.

Hmm, how would one check that? The output is to fast for me ;-) Seriously, how should one check this?

Thanks for your help,
Michael