routing issue
Julian Elischer
julian at freebsd.org
Fri Mar 4 19:17:04 UTC 2016
On 3/03/2016 2:38 AM, Pakhom Golynga wrote:
> Hello all!
>
> Please help me to investigate this issue.
> I have problem on 10.2-RELEASE-p12 with multiple network interfaces
> and PF (rules, NAT)
> # ifconfig
> <--cut-->
> em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu
> 1500
> options=4209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,VLAN_HWTSO>
>
> ether 00:25:90:64:14:50
> inet 172.27.27.254 netmask 0xffffff00 broadcast 172.27.27.255
> inet 172.27.27.240 netmask 0xffffffff broadcast 172.27.27.240
> inet 172.27.27.252 netmask 0xffffffff broadcast 172.27.27.252
> inet 172.27.27.251 netmask 0xffffffff broadcast 172.27.27.251
> inet 172.27.27.250 netmask 0xffffffff broadcast 172.27.27.250
> inet 172.27.27.249 netmask 0xffffffff broadcast 172.27.27.249
> inet 172.27.27.248 netmask 0xffffffff broadcast 172.27.27.248
> inet 172.27.27.247 netmask 0xffffffff broadcast 172.27.27.247
> inet 172.27.27.246 netmask 0xffffffff broadcast 172.27.27.246
> inet 172.27.27.245 netmask 0xffffffff broadcast 172.27.27.245
> inet 172.27.27.244 netmask 0xffffffff broadcast 172.27.27.244
> inet 172.27.27.243 netmask 0xffffffff broadcast 172.27.27.243
> inet 172.27.27.242 netmask 0xffffffff broadcast 172.27.27.242
> inet 172.27.27.241 netmask 0xffffffff broadcast 172.27.27.241
> inet 172.27.27.239 netmask 0xffffffff broadcast 172.27.27.239
> inet 172.27.27.238 netmask 0xffffffff broadcast 172.27.27.238
> inet 172.27.27.237 netmask 0xffffffff broadcast 172.27.27.237
> inet 172.27.27.236 netmask 0xffffffff broadcast 172.27.27.236
> inet 172.27.27.235 netmask 0xffffffff broadcast 172.27.27.235
> media: Ethernet autoselect (1000baseT <full-duplex>)
> status: active
> em1: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST>
> metric 0 mtu 1500
> options=4209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,VLAN_HWTSO>
>
> ether 00:25:90:64:14:51
> inet 10.24.44.50 netmask 0xfffffc00 broadcast 10.24.47.255
> media: Ethernet autoselect (100baseTX <full-duplex>)
> status: active
> <--cut-->
>
> em0 - local network. Aliases on em0 for Jail's
> em1 - connected to ISP
>
> This configuration successfully operate a long time but recently
> (for no apparent reason) suddenly became manifest bug with routing.
> Regardless from uptime (sometimes 12 hours, and maybe a couple of
> days) after a certain time the traffic running between jails begins
> to route via em1 (instead lo0 (net.link.ether.inet.useloopback=1)):
this sounds much like another report we have seen.
you may be able to fix it by re-adding your routes..
The report I have seen suggested that some routes seem to stop being
effective after some period.
It's still not understood.
> # tcpdump -n -e -ttt -i em1 host 172.27.27.252 or host 172.27.27.247
> or host 172.27.27.248
> 00:00:00.054989 00:25:90:64:14:51 > 68:bd:ab:5f:ea:d8, ethertype
> IPv4 (0x0800), length 74: 172.27.27.247.24868 > 172.27.27.252.25:
> Flags [S], seq 2075254199, win 65535, options [mss 16344,nop,wscale
> 6,sackOK,TS val 161568673 ecr 0], length 0
> 00:00:00.318036 00:25:90:64:14:51 > 68:bd:ab:5f:ea:d8, ethertype
> IPv4 (0x0800), length 66: 172.27.27.247.80 > 172.27.27.248.60176:
> Flags [.], ack 772660872, win 1275, options [nop,nop,TS val
> 1737287913 ecr 161568991], length 0
> 00:00:00.036971 00:25:90:64:14:51 > 68:bd:ab:5f:ea:d8, ethertype
> IPv4 (0x0800), length 66: 172.27.27.247.80 > 172.27.27.248.33080:
> Flags [.], ack 1, win 1275, options [nop,nop,TS val 1701623130 ecr
> 161568238], length 0
> 00:00:00.050001 00:25:90:64:14:51 > 68:bd:ab:5f:ea:d8, ethertype
> IPv4 (0x0800), length 66: 172.27.27.247.80 > 172.27.27.248.40200:
> Flags [.], ack 1, win 1275, options [nop,nop,TS val 1477098721 ecr
> 161568288], length 0
> 00:00:02.794992 00:25:90:64:14:51 > 68:bd:ab:5f:ea:d8, ethertype
> IPv4 (0x0800), length 74: 172.27.27.247.24868 > 172.27.27.252.25:
> Flags [S], seq 2075254199, win 65535, options [mss 16344,nop,wscale
> 6,sackOK,TS val 161571873 ecr 0], length 0
> 00:00:03.312026 00:25:90:64:14:51 > 68:bd:ab:5f:ea:d8, ethertype
> IPv4 (0x0800), length 74: 172.27.27.247.29581 > 172.27.27.252.25:
> Flags [S], seq 1217253021, win 65535, options [mss 16344,nop,wscale
> 6,sackOK,TS val 161575185 ecr 0], length 0
> 00:00:02.887982 00:25:90:64:14:51 > 68:bd:ab:5f:ea:d8, ethertype
> IPv4 (0x0800), length 74: 172.27.27.247.24868 > 172.27.27.252.25:
> Flags [S], seq 2075254199, win 65535, options [mss 16344,nop,wscale
> 6,sackOK,TS val 161578073 ecr 0], length 0
> 00:00:00.111989 00:25:90:64:14:51 > 68:bd:ab:5f:ea:d8, ethertype
> IPv4 (0x0800), length 74: 172.27.27.247.29581 > 172.27.27.252.25:
> Flags [S], seq 1217253021, win 65535, options [mss 16344,nop,wscale
> 6,sackOK,TS val 161578185 ecr 0], length 0
> 00:00:03.471007 00:25:90:64:14:51 > 68:bd:ab:5f:ea:d8, ethertype
> IPv4 (0x0800), length 108: 172.27.27.247.80 > 172.27.27.248.26056:
> Flags [FP.], seq 1448:1490, ack 1, win 1275, options [nop,nop,TS val
> 2142270801 ecr 161575507], length 42
>
> In the same time:
> # route -vn get -host
> RTA_DST: inet 172.27.27.247; RTA_IFP: link ; RTM_GET: Report
> Metrics: len 224, pid: 0, seq 1, errno 0,
> flags:<UP,GATEWAY,HOST,STATIC>
> locks: inits:
> sockaddrs: <DST,IFP>
> 172.27.27.247 link#0
> route to: 172.27.27.247
> destination: 172.27.27.247
> fib: 0
> interface: lo0
> flags: <UP,HOST,DONE,STATIC>
> recvpipe sendpipe ssthresh rtt,msec mtu weight expire
> 0 0 0 0 16384 1 0
>
> locks: inits:
> sockaddrs: <DST,GATEWAY,IFP,IFA>
> 172.27.27.247 link#4 lo0 127.0.0.1
> RTA_DST: inet 172.27.27.250; RTA_IFP: link ; RTM_GET: Report
> Metrics: len 224, pid: 0, seq 1, errno 0,
> flags:<UP,GATEWAY,HOST,STATIC>
> locks: inits:
> sockaddrs: <DST,IFP>
> 172.27.27.250 link#0
> route to: 172.27.27.250
> destination: 172.27.27.250
> fib: 0
> interface: lo0
> flags: <UP,HOST,DONE,STATIC>
> recvpipe sendpipe ssthresh rtt,msec mtu weight expire
> 0 0 0 0 16384 1 0
>
> locks: inits:
> sockaddrs: <DST,GATEWAY,IFP,IFA>
> 172.27.27.250 link#4 lo0 127.0.0.1
> RTA_DST: inet 172.27.27.252; RTA_IFP: link ; RTM_GET: Report
> Metrics: len 224, pid: 0, seq 1, errno 0,
> flags:<UP,GATEWAY,HOST,STATIC>
> locks: inits:
> sockaddrs: <DST,IFP>
> 172.27.27.252 link#0
> route to: 172.27.27.252
> destination: 172.27.27.252
> fib: 0
> interface: lo0
> flags: <UP,HOST,DONE,STATIC>
> recvpipe sendpipe ssthresh rtt,msec mtu weight expire
> 0 0 0 0 16384 1 0
>
> locks: inits:
> sockaddrs: <DST,GATEWAY,IFP,IFA>
> 172.27.27.252 link#4 lo0 127.0.0.1
>
>
> # netstat -rn
> Routing tables
>
> Internet:
> Destination Gateway Flags Netif Expire
> default 10.24.44.1 UGS em1
> 10.24.44.0/22 link#5 U em1
> 10.24.44.50 link#5 UHS lo0
> 127.0.0.1 link#7 UH lo0
> 172.27.27.0/24 link#4 U em0
> 172.27.27.235 link#4 UHS lo0
> 172.27.27.235/32 link#4 U em0
> 172.27.27.236 link#4 UHS lo0
> 172.27.27.236/32 link#4 U em0
> 172.27.27.237 link#4 UHS lo0
> 172.27.27.237/32 link#4 U em0
> 172.27.27.238 link#4 UHS lo0
> 172.27.27.238/32 link#4 U em0
> 172.27.27.239 link#4 UHS lo0
> 172.27.27.239/32 link#4 U em0
> 172.27.27.240 link#4 UHS lo0
> 172.27.27.240/32 link#4 U em0
> 172.27.27.241 link#4 UHS lo0
> 172.27.27.241/32 link#4 U em0
> 172.27.27.242 link#4 UHS lo0
> 172.27.27.242/32 link#4 U em0
> 172.27.27.243 link#4 UHS lo0
> 172.27.27.243/32 link#4 U em0
> 172.27.27.244 link#4 UHS lo0
> 172.27.27.244/32 link#4 U em0
> 172.27.27.245 link#4 UHS lo0
> 172.27.27.245/32 link#4 U em0
> 172.27.27.246 link#4 UHS lo0
> 172.27.27.246/32 link#4 U em0
> 172.27.27.247 link#4 UHS lo0
> 172.27.27.247/32 link#4 U em0
> 172.27.27.248 link#4 UHS lo0
> 172.27.27.248/32 link#4 U em0
> 172.27.27.249 link#4 UHS lo0
> 172.27.27.249/32 link#4 U em0
> 172.27.27.250 link#4 UHS lo0
> 172.27.27.250/32 link#4 U em0
> 172.27.27.251 link#4 UHS lo0
> 172.27.27.251/32 link#4 U em0
> 172.27.27.252 link#4 UHS lo0
> 172.27.27.252/32 link#4 U em0
> 172.27.27.254 link#4 UHS lo0
> 172.27.28.0/24 link#8 U bridge0
> 172.27.28.254 link#8 UHS lo0
> 172.27.30.0/24 172.27.30.2 UGS tun0
> 172.27.30.1 link#12 UHS lo0
> 172.27.30.2 link#12 UH tun0
>
> After the reboot everything works well for some time.
>
> Thanks!
> _______________________________________________
> freebsd-net at freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-net
> To unsubscribe, send any mail to "freebsd-net-unsubscribe at freebsd.org"
>
More information about the freebsd-net
mailing list