tcp window scaling + syn cookies problem

[Hooman Fazaeli]

Hi,

In our network, Windows clients connect to internet via our custom developed transparent
tcp proxy (running on 7.3). Things work fine, except that _sometimes_ downloads from the
some windows clients become very slow. To debug the problem, we inspected a few packet traces and
found out that the problem happens because the proxy TCP stack forgets about client's window
scale factor, as illustrated in the following packet trace (it is for a download from
ftp.freebsd.org site. x.y.z.y is a windows 8):

1. 15:09:32.765713 IP (tos 0x0, ttl 63, id 16510, offset 0, flags [DF], proto TCP (6), length 52)
     x.y.z.y.57430 > 96.47.72.72.80: S, cksum 0x8343 (correct), 1530161492:1530161492(0) win 8192 <mss 1460,nop,_wscale 8_,nop,nop,sackOK>

2. 15:09:32.765729 IP (tos 0x0, ttl 64, id 55869, offset 0, flags [none], proto TCP (6), length 52)
     96.47.72.72.80 > x.y.z.y.57430: S, cksum 0xe2c0 (correct), 503882603:503882603(0) ack 1530161493 win 65535 <mss 1460,nop,wscale 7,sackOK,eol>

3. 15:09:32.766071 IP (tos 0x0, ttl 63, id 16511, offset 0, flags [DF], proto TCP (6), length 40)
     x.y.z.y.57430 > 96.47.72.72.80: ., cksum 0x2192 (correct), ack 1 win 256

4. 15:09:32.770074 IP (tos 0x0, ttl 63, id 16512, offset 0, flags [DF], proto TCP (6), length 408)
     x.y.z.y.57430 > 96.47.72.72.80: P, cksum 0x259c (correct), 1:369(368) ack 1 win 256

5. 15:09:32.869286 IP (tos 0x0, ttl 64, id 57834, offset 0, flags [none], proto TCP (6), length 40)
     96.47.72.72.80 > x.y.z.y.57430: ., cksum 0x2122 (correct), ack 369 win 65535

6. 15:09:33.180983 IP (tos 0x0, ttl 64, id 64495, offset 0, flags [none], proto TCP (6), length 296)
     96.47.72.72.80 > x.y.z.y.57430: ., cksum 0xbd5a (correct), 1:257(256) ack 369 win 65535

7. 15:09:33.231475 IP (tos 0x0, ttl 63, id 16513, offset 0, flags [DF], proto TCP (6), length 40)
     x.y.z.y.57430 > 96.47.72.72.80: ., cksum 0x1f23 (correct), ack 257 win 255

8. 15:09:33.231494 IP (tos 0x0, ttl 64, id 248, offset 0, flags [none], proto TCP (6), length 295)
     96.47.72.72.80 > x.y.z.y.57430: ., cksum 0xc9b6 (correct), 257:512(255) ack 369 win 65535

9. 15:09:33.282256 IP (tos 0x0, ttl 63, id 16514, offset 0, flags [DF], proto TCP (6), length 40)
     x.y.z.y.57430 > 96.47.72.72.80: ., cksum 0x1e25 (correct), ack 512 win 254

10. 15:09:33.282279 IP (tos 0x0, ttl 64, id 1283, offset 0, flags [none], proto TCP (6), length 294)
     96.47.72.72.80 > x.y.z.y.57430: ., cksum 0x1e25 (correct), 512:766(254) ack 369 win 65535

11. 15:09:33.333006 IP (tos 0x0, ttl 63, id 16515, offset 0, flags [DF], proto TCP (6), length 40)
     x.y.z.y.57430 > 96.47.72.72.80: ., cksum 0x1d28 (correct), ack 766 win 253

12. 15:09:33.333023 IP (tos 0x0, ttl 64, id 2520, offset 0, flags [none], proto TCP (6), length 293)
     96.47.72.72.80 > x.y.z.y.57430: ., cksum 0x1d28 (correct), 766:1019(253) ack 369 win 65535

13. 15:09:33.383926 IP (tos 0x0, ttl 63, id 16516, offset 0, flags [DF], proto TCP (6), length 40)
     x.y.z.y.57430 > 96.47.72.72.80: ., cksum 0x1c2c (correct), ack 1019 win 252

As can be seen, the client advertises a window scale factor of 8 and then correctly
sets packet's window size based on the advertised factor. But the proxy
seems to forget about client's scale factor and sends as much data as the
client's unscaled window size sent in a previous ACK.

Now, setting 'net.inet.tcp.syncookies' to zero obviously seems to fix the problem
and the download speed becomes as expected.

Is this bad interaction between window scaling and syn cookies
a known problem? Why it happens? Has it been fixed in later freebsd version?

Thanks in advance.

-- 
Best regards
Hooman Fazaeli