LRO causing stretch ACK violations interacts badly with delayed ACKing

[Colin Percival]

On 10/20/13 17:44, Julian Elischer wrote:
> On 10/18/13 6:03 AM, Colin Percival wrote:
>> I know {TSO, LRO, ACKing policy} has been discussed here recently, and I don't
>> want to rehash everything, but I'm seeing some very bad misbehaviour with LRO
>> and delayed ACKing turned on.
>>
>> Running 'fetch -o /dev/null https://www.amazon.com/' on an EC2 instance running
> [...]
> is this just for -current?

Good question.  Turns out that it isn't -- on 9.2 I see a 95.5 ms delayed ACK:

> 00:00:00.000000 IP 10.142.129.245.59172 > 176.32.98.166.443: Flags [S], seq 3310207763, win 65535, options [mss 1460,nop,wscale 6,sackOK,TS val 292712 ecr 0], length 0
> 00:00:00.001031 IP 176.32.98.166.443 > 10.142.129.245.59172: Flags [S.], seq 3504196464, ack 3310207764, win 8190, options [mss 1460,nop,wscale 6], length 0
> 00:00:00.001139 IP 10.142.129.245.59172 > 176.32.98.166.443: Flags [.], ack 1, win 1026, length 0
> 00:00:00.002269 IP 176.32.98.166.443 > 10.142.129.245.59172: Flags [.], ack 1, win 127, length 0
> 00:00:00.002938 IP 10.142.129.245.59172 > 176.32.98.166.443: Flags [P.], seq 1:140, ack 1, win 1026, length 139
> 00:00:00.003815 IP 176.32.98.166.443 > 10.142.129.245.59172: Flags [.], seq 1:4097, ack 140, win 108, length 4096
> 00:00:00.099328 IP 10.142.129.245.59172 > 176.32.98.166.443: Flags [.], ack 4097, win 1026, length 0

but not on 9.1... although that might just be that LRO isn't happening:
> 00:00:00.000000 IP 10.148.177.92.48728 > 176.32.98.166.443: Flags [S], seq 2729946716, win 65535, options [mss 1460,nop,wscale 6,sackOK,TS val 64564 ecr 0], length 0
> 00:00:00.000722 IP 176.32.98.166.443 > 10.148.177.92.48728: Flags [S.], seq 595247561, ack 2729946717, win 8190, options [mss 1460,nop,wscale 6], length 0
> 00:00:00.000820 IP 10.148.177.92.48728 > 176.32.98.166.443: Flags [.], ack 1, win 1026, length 0
> 00:00:00.001998 IP 176.32.98.166.443 > 10.148.177.92.48728: Flags [.], ack 1, win 127, length 0
> 00:00:00.002716 IP 10.148.177.92.48728 > 176.32.98.166.443: Flags [P.], seq 1:140, ack 1, win 1026, length 139
> 00:00:00.003527 IP 176.32.98.166.443 > 10.148.177.92.48728: Flags [.], ack 140, win 108, length 0
> 00:00:00.003834 IP 176.32.98.166.443 > 10.148.177.92.48728: Flags [.], seq 1:1461, ack 140, win 108, length 1460
> 00:00:00.003850 IP 176.32.98.166.443 > 10.148.177.92.48728: Flags [.], seq 1461:2921, ack 140, win 108, length 1460
> 00:00:00.003870 IP 10.148.177.92.48728 > 176.32.98.166.443: Flags [.], ack 2921, win 981, length 0
> 00:00:00.003888 IP 176.32.98.166.443 > 10.148.177.92.48728: Flags [P.], seq 2921:4097, ack 140, win 108, length 1176
> 00:00:00.003973 IP 10.148.177.92.48728 > 176.32.98.166.443: Flags [.], ack 4097, win 1026, length 0

I can't find any changes in netfront.c or tcp_lro.c to explain why 9.1 and
9.2 are behaving differently -- anyone have any ideas?

-- 
Colin Percival
Security Officer Emeritus, FreeBSD | The power to serve
Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid