TCP socket handling lo[X], buggy in 9.x?
Matthew Grooms
mgrooms at shrew.net
Tue Jan 14 07:48:41 UTC 2014
All,
I am seeing weird issues with tcp connections when they are established
from an address that is assigned to a lo interface on 9.2-RELEASE. This
works fine on older FreeBSD hosts that I have tested ( like 7.1-RELEASE
and 8.2-RELEASE ) but appears to be broken in 9.2-RELEASE-p2. Here is an
example of the issue ...
host1 ( 9.2-RELEASE )
\
em0: inet 10.22.200.101/24
lo1: inet inet 172.16.1.1/32
host2 ( 8.2-RELEASE )
\
em0: inet 10.22.200.102/24
lo1: inet inet 172.18.1.1/32
... I add a route on host1 to reach 172.18.1.1/32 on host2 ...
[host1]# route add -host 172.18.1.1 10.22.200.102
... and add a route on host2 to reach 172.16.1.1/32 on host1 ...
[host2]# route add -host 172.16.1.1 10.22.200.101
... Then I use netcat to connect from host1 to host2. When I connect
from the local em0 interface to the remote em0 interface, everything is
good ...
[host1]# nc -s 10.22.200.101 10.22.200.102 1000
line1
line2
line3
line4
[host2]# nc -k -l 1000
line1
line2
line3
line4
... when I connect from the local em0 interface to the remote lo1
interface, everything looks good ( to lo1 on the 8.2 host ) ...
[host1]# nc -s 10.22.200.101 172.18.1.1 1000
line1
line2
line3
line4
[host2]# nc -k -l 1000
line1
line2
line3
line4
... When I connect from the local lo1 interface to the remote em0
interface, weird things start to happen ( from lo1 on the 9.2 host ) ...
[host1]# nc -s 172.16.1.1 10.22.200.102 1000
line1
line2
<connection spontaneously closes>
[host1]# nc -s 172.16.1.1 10.22.200.102 1000
line1
line2
<connection spontaneously closes>
[host1]# nc -s 172.16.1.1 10.22.200.102 1000
line1
line2
<connection spontaneously closes>
[host2]# nc -k -l 1000
line1
line1
line1
... Here is a comparison of the tcpdump output from scenario #1 vs
scenario #3. ...
tcpdump -ni em0 src or dst 10.22.200.102
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on em0, link-type EN10MB (Ethernet), capture size 65535 bytes
00:49:34.897822 IP 10.22.200.101.18401 > 10.22.200.102.1000: Flags [S],
seq 2109171603, win 65535, options [mss 1460,nop,wscale 6,sackOK,TS val
113771252 ecr 0], length 0
00:49:34.898133 IP 10.22.200.102.1000 > 10.22.200.101.18401: Flags [S.],
seq 1520138341, ack 2109171604, win 65535, options [mss 1460,nop,wscale
3,sackOK,TS val 763106128 ecr 113771252], length 0
00:49:34.898242 IP 10.22.200.101.18401 > 10.22.200.102.1000: Flags [.],
ack 1, win 1040, options [nop,nop,TS val 113771252 ecr 763106128], length 0
00:49:36.411982 IP 10.22.200.101.18401 > 10.22.200.102.1000: Flags [P.],
seq 1:7, ack 1, win 1040, options [nop,nop,TS val 113772775 ecr
763106128], length 6
00:49:36.502874 IP 10.22.200.102.1000 > 10.22.200.101.18401: Flags [.],
ack 7, win 8326, options [nop,nop,TS val 763106289 ecr 113772775], length 0
00:49:37.635691 IP 10.22.200.101.18401 > 10.22.200.102.1000: Flags [P.],
seq 7:13, ack 1, win 1040, options [nop,nop,TS val 113773992 ecr
763106289], length 6
00:49:37.733039 IP 10.22.200.102.1000 > 10.22.200.101.18401: Flags [.],
ack 13, win 8326, options [nop,nop,TS val 763106412 ecr 113773992], length 0
00:49:40.467889 IP 10.22.200.101.18401 > 10.22.200.102.1000: Flags [P.],
seq 13:19, ack 1, win 1040, options [nop,nop,TS val 113776831 ecr
763106412], length 6
00:49:40.562215 IP 10.22.200.102.1000 > 10.22.200.101.18401: Flags [.],
ack 19, win 8326, options [nop,nop,TS val 763106695 ecr 113776831], length 0
00:49:42.148281 IP 10.22.200.101.18401 > 10.22.200.102.1000: Flags [P.],
seq 19:25, ack 1, win 1040, options [nop,nop,TS val 113778511 ecr
763106695], length 6
00:49:42.242122 IP 10.22.200.102.1000 > 10.22.200.101.18401: Flags [.],
ack 25, win 8326, options [nop,nop,TS val 763106863 ecr 113778511], length 0
00:49:42.556244 IP 10.22.200.101.18401 > 10.22.200.102.1000: Flags [F.],
seq 25, ack 1, win 1040, options [nop,nop,TS val 113778912 ecr
763106863], length 0
00:49:42.556697 IP 10.22.200.102.1000 > 10.22.200.101.18401: Flags [.],
ack 26, win 8326, options [nop,nop,TS val 763106894 ecr 113778912], length 0
00:49:42.556722 IP 10.22.200.102.1000 > 10.22.200.101.18401: Flags [F.],
seq 1, ack 26, win 8326, options [nop,nop,TS val 763106894 ecr
113778912], length 0
00:49:42.556806 IP 10.22.200.101.18401 > 10.22.200.102.1000: Flags [.],
ack 2, win 1040, options [nop,nop,TS val 113778912 ecr 763106894], length 0
tcpdump -ni em0 src or dst 10.22.200.102
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on em0, link-type EN10MB (Ethernet), capture size 65535 bytes
00:52:02.358683 IP 172.16.1.1.10364 > 10.22.200.102.1000: Flags [S], seq
2289590664, win 65535, options [mss 1460,nop,wscale 6,sackOK,TS val
113918712 ecr 0], length 0
00:52:02.358896 IP 10.22.200.102.1000 > 172.16.1.1.10364: Flags [S.],
seq 2860805865, ack 2289590665, win 65535, options [mss 1460,nop,wscale
3,sackOK,TS val 2333279786 ecr 113918712], length 0
00:52:02.359010 IP 172.16.1.1.10364 > 10.22.200.102.1000: Flags [.], ack
1, win 1040, options [nop,nop,TS val 113918722 ecr 2333279786], length 0
00:52:02.359015 IP 172.16.1.1.10364 > 10.22.200.102.1000: Flags [R], seq
2289590665, win 0, length 0
00:52:04.224632 IP 172.16.1.1.10364 > 10.22.200.102.1000: Flags [P.],
seq 1:7, ack 1, win 1040, options [nop,nop,TS val 113920582 ecr
2333279786], length 6
00:52:04.322995 IP 10.22.200.102.1000 > 172.16.1.1.10364: Flags [.], ack
7, win 8326, options [nop,nop,TS val 2333279983 ecr 113920582], length 0
00:52:04.323026 IP 172.16.1.1.10364 > 10.22.200.102.1000: Flags [R], seq
2289590671, win 0, length 0
00:52:05.752341 IP 172.16.1.1.10364 > 10.22.200.102.1000: Flags [P.],
seq 7:13, ack 1, win 1040, options [nop,nop,TS val 113922112 ecr
2333279983], length 6
00:52:05.752595 IP 10.22.200.102.1000 > 172.16.1.1.10364: Flags [R], seq
2860805866, win 0, length 0
00:52:06.798407 IP 172.16.1.1.61952 > 10.22.200.102.1000: Flags [S], seq
3819912610, win 65535, options [mss 1460,nop,wscale 6,sackOK,TS val
113923152 ecr 0], length 0
00:52:06.798588 IP 10.22.200.102.1000 > 172.16.1.1.61952: Flags [S.],
seq 889829473, ack 3819912611, win 65535, options [mss 1460,nop,wscale
3,sackOK,TS val 1369790257 ecr 113923152], length 0
00:52:06.798657 IP 172.16.1.1.61952 > 10.22.200.102.1000: Flags [.], ack
1, win 1040, options [nop,nop,TS val 113923152 ecr 1369790257], length 0
00:52:06.798727 IP 172.16.1.1.61952 > 10.22.200.102.1000: Flags [R], seq
3819912611, win 0, length 0
00:52:08.328904 IP 172.16.1.1.61952 > 10.22.200.102.1000: Flags [P.],
seq 1:7, ack 1, win 1040, options [nop,nop,TS val 113924692 ecr
1369790257], length 6
00:52:08.422462 IP 10.22.200.102.1000 > 172.16.1.1.61952: Flags [.], ack
7, win 8326, options [nop,nop,TS val 1369790267 ecr 113924692], length 0
00:52:08.422542 IP 172.16.1.1.61952 > 10.22.200.102.1000: Flags [R], seq
3819912617, win 0, length 0
00:52:09.968562 IP 172.16.1.1.61952 > 10.22.200.102.1000: Flags [P.],
seq 7:13, ack 1, win 1040, options [nop,nop,TS val 113926332 ecr
1369790267], length 6
00:52:09.968846 IP 10.22.200.102.1000 > 172.16.1.1.61952: Flags [R], seq
889829474, win 0, length 0
00:52:12.869940 IP 172.16.1.1.35530 > 10.22.200.102.1000: Flags [S], seq
197894496, win 65535, options [mss 1460,nop,wscale 6,sackOK,TS val
113929232 ecr 0], length 0
00:52:12.870144 IP 10.22.200.102.1000 > 172.16.1.1.35530: Flags [S.],
seq 4171403115, ack 197894497, win 65535, options [mss 1460,nop,wscale
3,sackOK,TS val 2442049100 ecr 113929232], length 0
00:52:12.870211 IP 172.16.1.1.35530 > 10.22.200.102.1000: Flags [.], ack
1, win 1040, options [nop,nop,TS val 113929232 ecr 2442049100], length 0
00:52:12.870331 IP 172.16.1.1.35530 > 10.22.200.102.1000: Flags [R], seq
197894497, win 0, length 0
00:52:14.144771 IP 172.16.1.1.35530 > 10.22.200.102.1000: Flags [P.],
seq 1:7, ack 1, win 1040, options [nop,nop,TS val 113930508 ecr
2442049100], length 6
00:52:14.241838 IP 10.22.200.102.1000 > 172.16.1.1.35530: Flags [.], ack
7, win 8326, options [nop,nop,TS val 2442049110 ecr 113930508], length 0
00:52:14.241864 IP 172.16.1.1.35530 > 10.22.200.102.1000: Flags [R], seq
197894503, win 0, length 0
00:52:15.152672 IP 172.16.1.1.35530 > 10.22.200.102.1000: Flags [P.],
seq 7:13, ack 1, win 1040, options [nop,nop,TS val 113931512 ecr
2442049110], length 6
00:52:15.153065 IP 10.22.200.102.1000 > 172.16.1.1.35530: Flags [R], seq
4171403116, win 0, length 0
It looks like the 9.2 host is sending a reset before sending every PDU.
Weird. In any case, this is really easy to re-produce. Anyone have an
idea as to why this started happening? I didn't try FreeBSD 10, but I
suspect the problem is the same.
Thanks,
-Matthew
More information about the freebsd-net
mailing list