tcp window scaling + syn cookies problem

[Hooman Fazaeli]

On 2016-03-07 4:26 PM, Hooman Fazaeli wrote:
>
> Hi,
>
> In our network, Windows clients connect to internet via our custom developed transparent
> tcp proxy (running on 7.3). Things work fine, except that _sometimes_ downloads from the
> some windows clients become very slow. To debug the problem, we inspected a few packet traces and
> found out that the problem happens because the proxy TCP stack forgets about client's window
> scale factor, as illustrated in the following packet trace (it is for a download from
> ftp.freebsd.org site. x.y.z.y is a windows 8):
>
> 1. 15:09:32.765713 IP (tos 0x0, ttl 63, id 16510, offset 0, flags [DF], proto TCP (6), length 52)
>     x.y.z.y.57430 > 96.47.72.72.80: S, cksum 0x8343 (correct), 1530161492:1530161492(0) win 8192 <mss 1460,nop,_wscale 8_,nop,nop,sackOK>
>
> 2. 15:09:32.765729 IP (tos 0x0, ttl 64, id 55869, offset 0, flags [none], proto TCP (6), length 52)
>     96.47.72.72.80 > x.y.z.y.57430: S, cksum 0xe2c0 (correct), 503882603:503882603(0) ack 1530161493 win 65535 <mss 1460,nop,wscale 7,sackOK,eol>
>
> 3. 15:09:32.766071 IP (tos 0x0, ttl 63, id 16511, offset 0, flags [DF], proto TCP (6), length 40)
>     x.y.z.y.57430 > 96.47.72.72.80: ., cksum 0x2192 (correct), ack 1 win 256
>
> 4. 15:09:32.770074 IP (tos 0x0, ttl 63, id 16512, offset 0, flags [DF], proto TCP (6), length 408)
>     x.y.z.y.57430 > 96.47.72.72.80: P, cksum 0x259c (correct), 1:369(368) ack 1 win 256
>
> 5. 15:09:32.869286 IP (tos 0x0, ttl 64, id 57834, offset 0, flags [none], proto TCP (6), length 40)
>     96.47.72.72.80 > x.y.z.y.57430: ., cksum 0x2122 (correct), ack 369 win 65535
>
> 6. 15:09:33.180983 IP (tos 0x0, ttl 64, id 64495, offset 0, flags [none], proto TCP (6), length 296)
>     96.47.72.72.80 > x.y.z.y.57430: ., cksum 0xbd5a (correct), 1:257(256) ack 369 win 65535
>
> 7. 15:09:33.231475 IP (tos 0x0, ttl 63, id 16513, offset 0, flags [DF], proto TCP (6), length 40)
>     x.y.z.y.57430 > 96.47.72.72.80: ., cksum 0x1f23 (correct), ack 257 win 255
>
> 8. 15:09:33.231494 IP (tos 0x0, ttl 64, id 248, offset 0, flags [none], proto TCP (6), length 295)
>     96.47.72.72.80 > x.y.z.y.57430: ., cksum 0xc9b6 (correct), 257:512(255) ack 369 win 65535
>
> 9. 15:09:33.282256 IP (tos 0x0, ttl 63, id 16514, offset 0, flags [DF], proto TCP (6), length 40)
>     x.y.z.y.57430 > 96.47.72.72.80: ., cksum 0x1e25 (correct), ack 512 win 254
>
> 10. 15:09:33.282279 IP (tos 0x0, ttl 64, id 1283, offset 0, flags [none], proto TCP (6), length 294)
>     96.47.72.72.80 > x.y.z.y.57430: ., cksum 0x1e25 (correct), 512:766(254) ack 369 win 65535
>
> 11. 15:09:33.333006 IP (tos 0x0, ttl 63, id 16515, offset 0, flags [DF], proto TCP (6), length 40)
>     x.y.z.y.57430 > 96.47.72.72.80: ., cksum 0x1d28 (correct), ack 766 win 253
>
> 12. 15:09:33.333023 IP (tos 0x0, ttl 64, id 2520, offset 0, flags [none], proto TCP (6), length 293)
>     96.47.72.72.80 > x.y.z.y.57430: ., cksum 0x1d28 (correct), 766:1019(253) ack 369 win 65535
>
> 13. 15:09:33.383926 IP (tos 0x0, ttl 63, id 16516, offset 0, flags [DF], proto TCP (6), length 40)
>     x.y.z.y.57430 > 96.47.72.72.80: ., cksum 0x1c2c (correct), ack 1019 win 252
>
> As can be seen, the client advertises a window scale factor of 8 and then correctly
> sets packet's window size based on the advertised factor. But the proxy
> seems to forget about client's scale factor and sends as much data as the
> client's unscaled window size sent in a previous ACK.
>
> Now, setting 'net.inet.tcp.syncookies' to zero obviously seems to fix the problem
> and the download speed becomes as expected.
>
> Is this bad interaction between window scaling and syn cookies
> a known problem? Why it happens? Has it been fixed in later freebsd version?
>
> Thanks in advance.
>

A few minutes after posting, I found the following thread which describes an
exact duplicate of our problem : https://lists.freebsd.org/pipermail/freebsd-net/2013-February/034519.html



-- 
Best regards
Hooman Fazaeli