RE: Too aggressive TCP ACKs
- In reply to: Zhenlei Huang : "Too aggressive TCP ACKs"
- Go to: [ bottom of page ] [ top of archives ] [ this month ]
Date: Thu, 27 Oct 2022 08:12:08 UTC
To come back to this.
With TSO / LRO disabled, FBSD is behaving per RFC by acking every other data packet, or delaying an ACK (e.g when data stops on an “uneven” packet) after a short delay (delayed ACKs).
If you want to see different ACK ratios, and also higher gigabit throughput rates for a single session, maybe test with the FBSD RACK stack
https://klarasystems.com/articles/using-the-freebsd-rack-tcp-stack/
sysctl net.inet.tcp.functions_default=rack
Richard
From: owner-freebsd-net@freebsd.org <owner-freebsd-net@freebsd.org> On Behalf Of Zhenlei Huang
Sent: Freitag, 21. Oktober 2022 16:19
To: freebsd-net@freebsd.org
Subject: Too aggressive TCP ACKs
NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe.
Hi,
While I was repeating https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=258755, I observed a
strange behavior. The TCP ACKs from FreeBSD host are too aggressive.
My setup is simple:
A B
[ MacOS ] <====> [ FreeBSD VM ]
192.168.120.1 192.168.12.134 (disable tso and lro)
While A <--- B, i.e. A as server and B as client, the packets rate looks good.
One session on B:
root@:~ # iperf3 -c 192.168.120.1 -b 10m
Connecting to host 192.168.120.1, port 5201
[ 5] local 192.168.120.134 port 54459 connected to 192.168.120.1 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 1.25 MBytes 10.5 Mbits/sec 0 257 KBytes
[ 5] 1.00-2.00 sec 1.25 MBytes 10.5 Mbits/sec 0 257 KBytes
[ 5] 2.00-3.00 sec 1.12 MBytes 9.44 Mbits/sec 0 257 KBytes
[ 5] 3.00-4.00 sec 1.25 MBytes 10.5 Mbits/sec 0 257 KBytes
[ 5] 4.00-5.00 sec 1.12 MBytes 9.44 Mbits/sec 0 257 KBytes
[ 5] 5.00-6.00 sec 1.25 MBytes 10.5 Mbits/sec 0 257 KBytes
[ 5] 6.00-7.00 sec 1.12 MBytes 9.44 Mbits/sec 0 257 KBytes
[ 5] 7.00-8.00 sec 1.25 MBytes 10.5 Mbits/sec 0 257 KBytes
[ 5] 8.00-9.00 sec 1.12 MBytes 9.44 Mbits/sec 0 257 KBytes
[ 5] 9.00-10.00 sec 1.25 MBytes 10.5 Mbits/sec 0 257 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 12.0 MBytes 10.1 Mbits/sec 0 sender
[ 5] 0.00-10.00 sec 12.0 MBytes 10.1 Mbits/sec receiver
iperf Done.
Another session on B:
root@:~ # netstat -w 1 -I vmx0
input vmx0 output
packets errs idrops bytes packets errs bytes colls
0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0
342 0 0 22600 526 0 775724 0
150 0 0 9900 851 0 1281454 0
109 0 0 7194 901 0 1357850 0
126 0 0 8316 828 0 1246632 0
122 0 0 8052 910 0 1370780 0
109 0 0 7194 819 0 1233702 0
120 0 0 7920 910 0 1370780 0
110 0 0 7260 819 0 1233702 0
123 0 0 8118 910 0 1370780 0
109 0 0 7194 819 0 1233702 0
73 0 0 5088 465 0 686342 0
0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0
================================================================
While A ---> B, i.e. A as client and B as server, the ACKs sent from B looks strange.
Session on A:
% iperf3 -c 192.168.120.134 -b 10m
Connecting to host 192.168.120.134, port 5201
[ 5] local 192.168.120.1 port 52370 connected to 192.168.120.134 port 5201
[ ID] Interval Transfer Bitrate
[ 5] 0.00-1.00 sec 1.25 MBytes 10.5 Mbits/sec
[ 5] 1.00-2.00 sec 1.25 MBytes 10.5 Mbits/sec
[ 5] 2.00-3.00 sec 1.12 MBytes 9.44 Mbits/sec
[ 5] 3.00-4.00 sec 1.25 MBytes 10.5 Mbits/sec
[ 5] 4.00-5.00 sec 1.12 MBytes 9.44 Mbits/sec
[ 5] 5.00-6.00 sec 1.25 MBytes 10.5 Mbits/sec
[ 5] 6.00-7.00 sec 1.12 MBytes 9.44 Mbits/sec
[ 5] 7.00-8.00 sec 1.25 MBytes 10.5 Mbits/sec
[ 5] 8.00-9.00 sec 1.12 MBytes 9.44 Mbits/sec
[ 5] 9.00-10.00 sec 1.25 MBytes 10.5 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate
[ 5] 0.00-10.00 sec 12.0 MBytes 10.1 Mbits/sec sender
[ 5] 0.00-10.00 sec 12.0 MBytes 10.1 Mbits/sec receiver
iperf Done.
Session on B:
root@:~ # netstat -w 1 -I vmx0
input vmx0 output
packets errs idrops bytes packets errs bytes colls
0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0
649 0 0 960562 330 0 21800 0
819 0 0 1233702 415 0 27390 0
910 0 0 1370780 459 0 30294 0
819 0 0 1233702 415 0 27390 0
910 0 0 1370780 459 0 30294 0
910 0 0 1370780 460 0 30360 0
819 0 0 1233702 414 0 27324 0
910 0 0 1370780 460 0 30360 0
819 0 0 1233702 414 0 27324 0
910 0 0 1370780 460 0 30360 0
285 0 0 412287 147 0 9981 0
0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0
The ACK packets replied from B (the FreeBSD VM) are too aggressive. They are
about one half of TCP packets received from A.
I've tested with different bitrates, from 10m to 300m, all behave the same.
Tested with baremetal FreeBSD 13.1 Box as B (with intel em driver), the
bitrates is 1g, also behaves the same.
Also tried different FreeBSD versions, 11.4, 12.3, stable/13 and current/14 all
behave the same.
My question is, is that the expected behavior of current default TCP stack?
Best regards,
Zhenlei