Router on 6.0-stable fails to route tcp packets due to NAT??
malfunction
Oleg Tarasov
subscriber at osk.com.ua
Mon Dec 26 03:44:40 PST 2005
Hello, all
SYSTEM DESCRIPTION
I have built a production system based on FreeBSD 6.0-stable. The main
Internet connection is established using mpd 3.18 which is started by
attached script "mpd". It is rcorder'ed similar to ppp-user.
mpd configuration is attached in mpd.conf and mpd.links. Shortly, ng0
is a PPPoE connection on rl1 interface.
By the way user ppp failed to work with PPPoE connection correctly
usually causing "No buffer space available" error which caused all
network connections to stop working. Manual restart of ppp helped but
it is quite unacceptable for production system. I attach ppp.conf
Firewall is configured to manually divert packets to natd. I attach
rc.firewall which was simplifyed to a minimum of functions for test
purposes.
natd is configured using the following config file:
===============================================================
log no
use_sockets yes
same_ports yes
interface ng0
unregistered_only yes
log_ipfw_denied yes
log_denied yes
===============================================================
I attach kernel configuration file used to compile it.
Here is output of ifconfig:
===============================================================
rl0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
options=8<VLAN_MTU>
inet 192.168.82.253 netmask 0xffffff00 broadcast 192.168.82.255
ether 00:30:4f:1c:ed:19
media: Ethernet autoselect (100baseTX <full-duplex>)
status: active
rl1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
options=8<VLAN_MTU>
ether 00:30:4f:1c:ed:17
media: Ethernet autoselect (100baseTX <full-duplex>)
status: active
plip0: flags=108810<POINTOPOINT,SIMPLEX,MULTICAST,NEEDSGIANT> mtu 1500
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 16384
inet 127.0.0.1 netmask 0xff000000
ng0: flags=88d1<UP,POINTOPOINT,RUNNING,NOARP,SIMPLEX,MULTICAST> mtu 1492
inet my.ip.add.ress --> prov.ip.add.ress netmask 0xffffffff
===============================================================
Here is output of netstat -rn:
===============================================================
Destination Gateway Flags Refs Use Netif Expire
default prov.ip.add.ress UGS 0 512334 ng0
my.ip.add.ress lo0 UHS 0 2426 lo0
127.0.0.1 127.0.0.1 UH 0 21881 lo0
192.168.82 link#1 UC 0 0 rl0
192.168.82.253 00:30:4f:1c:ed:19 UHLW 1 1162 lo0
prov.ip.add.ress my.ip.add.ress UH 1 0 ng0
===============================================================
Windows client configuration:
===============================================================
inet 192.168.82.111 netmask 255.255.255.0 192.168.82.253
===============================================================
Windows client routing table
===============================================================
0.0.0.0 0.0.0.0 192.168.82.253 192.168.82.111 30
127.0.0.0 255.0.0.0 127.0.0.1 127.0.0.1 1
192.168.0.0 255.255.255.0 192.168.82.204 192.168.82.111 1
192.168.82.0 255.255.255.0 192.168.82.111 192.168.82.111 30
192.168.82.111 255.255.255.255 127.0.0.1 127.0.0.1 30
192.168.82.255 255.255.255.255 192.168.82.111 192.168.82.111 30
224.0.0.0 240.0.0.0 192.168.82.111 192.168.82.111 30
255.255.255.255 255.255.255.255 192.168.82.111 192.168.82.111 1
Default gateway: 192.168.82.253
===============================================================
The system has SQUID, mail, ftp systems and usually direct packet
routing was not used so the problem was located after a month of usage
of the system.
PROBLEM DESCRIPTION
I have a number of Windows XP clients in the network which are
configured to use This machine as a default gateway. Any icmp packets
to Internet work quite normal. Web worked normally too but when using
proxy, so packet routing is not used for that.
The problem was first encountered when trying to play online game
which did not use proxy. Later it was confirmed when trying to serf
the Web with usage of proxy turned off.
Problem is that almost all data is not transmitted normally using tcp
connections. For example trying to open www.gnome.org fails completely
but packet flow seems to be normal. The most strange thing is that
this problem occurs only on some clients when other ones work quite
fine!!! From malfunctioning machines some sites can be opened too!!!
Some sites can be opened partitially - some parts like pictures can
fail to open.
You can say - "How can we be sure that you client machines are
configured normally?" - I am system administrator for some years and
have plenty of servers and clients confugured by my hands. Also I have
a production system based on 5.4p5 which is configured similarly to
this one but using kernel ppp for internet connection - but that one
had no problems.
Everything in the LAN works perfectly. Also everything going through
proxy also works fine. Any connection made directly from server has no
problems. This makes me think the problem is in routing or NAT.
For test purposes I have reinstalled my own client machine (which also
has the problem described above) from scratch - no result. I changed
network card, changed IP address - no positive result.
From all above I make a conclusion that possible reason is in the NAT
malfunction. Or I dont know what...
Here is the dump on both interfaces ng0 and rl0 which are Internet and
LAN interfaces. I try to open www.gnome.org and I see this:
tcpdump on ng0
===============================================================
09:55:13.757127 IP (tos 0x0, ttl 127, id 56112, offset 0, flags [DF], proto: TCP (6), length: 48) piramida.com.ua.1140 > window.gnome.org.http: S, cksum 0x2b0b (correct), 687058407:687058407(0) win 16384 <mss 1460,nop,nop,sackOK>
09:55:13.982233 IP (tos 0x0, ttl 47, id 0, offset 0, flags [DF], proto: TCP (6), length: 48) window.gnome.org.http > piramida.com.ua.1140: S, cksum 0x6f48 (correct), 3785163588:3785163588(0) ack 687058408 win 5840 <mss 1460,nop,nop,sackOK>
09:55:13.982616 IP (tos 0x0, ttl 127, id 56115, offset 0, flags [DF], proto: TCP (6), length: 40) piramida.com.ua.1140 > window.gnome.org.http: ., cksum 0x6e6c (correct), ack 1 win 17520
09:55:13.982774 IP (tos 0x0, ttl 127, id 56116, offset 0, flags [DF], proto: TCP (6), length: 322) piramida.com.ua.1140 > window.gnome.org.http: P 1:283(282) ack 1 win 17520
09:55:14.219491 IP (tos 0x0, ttl 47, id 58466, offset 0, flags [DF], proto: TCP (6), length: 40) window.gnome.org.http > piramida.com.ua.1140: ., cksum 0x98a2 (correct), ack 283 win 6432
09:55:59.300589 IP (tos 0x0, ttl 127, id 62999, offset 0, flags [DF], proto: TCP (6), length: 40) piramida.com.ua.1140 > window.gnome.org.http: R, cksum 0xb1be (correct), 283:283(0) ack 1 win 0
09:55:59.417698 IP (tos 0x0, ttl 64, id 36993, offset 0, flags [none], proto: TCP (6), length: 40) 192.168.82.111.1140 > window.gnome.org.http: ., cksum 0x58ec (correct), ack 3785163589 win 0
^^^^^^ ^^^^^^^^^^^^^^^^^^
!!!!!! !!!!!!!!!!!!!!!!!!
===============================================================
tcpdump on rl0
===============================================================
09:55:13.756938 IP (tos 0x0, ttl 128, id 56112, offset 0, flags [DF], proto: TCP (6), length: 48) 192.168.82.111.1140 > window.gnome.org.http: S, cksum 0xd233 (correct), 687058407:687058407(0) win 16384 <mss 1460,nop,nop,sackOK>
09:55:13.982399 IP (tos 0x0, ttl 46, id 0, offset 0, flags [DF], proto: TCP (6), length: 48) window.gnome.org.http > 192.168.82.111.1140: S, cksum 0x1671 (correct), 3785163588:3785163588(0) ack 687058408 win 5840 <mss 1460,nop,nop,sackOK>
09:55:13.982538 IP (tos 0x0, ttl 128, id 56115, offset 0, flags [DF], proto: TCP (6), length: 40) 192.168.82.111.1140 > window.gnome.org.http: ., cksum 0x1595 (correct), ack 1 win 17520
09:55:13.982719 IP (tos 0x0, ttl 128, id 56116, offset 0, flags [DF], proto: TCP (6), length: 322) 192.168.82.111.1140 > window.gnome.org.http: P 1:283(282) ack 1 win 17520
09:55:14.219666 IP (tos 0x0, ttl 46, id 58466, offset 0, flags [DF], proto: TCP (6), length: 40) window.gnome.org.http > 192.168.82.111.1140: ., cksum 0x3fcb (correct), ack 283 win 6432
09:55:59.300444 IP (tos 0x0, ttl 128, id 62999, offset 0, flags [DF], proto: TCP (6), length: 40) 192.168.82.111.1140 > window.gnome.org.http: R, cksum 0x58e7 (correct), 283:283(0) ack 1 win 0
09:55:59.417786 IP (tos 0x0, ttl 64, id 36994, offset 0, flags [none], proto: TCP (6), length: 40) window.gnome.org.http > 192.168.82.111.1140: ., cksum 0x58ec (correct), ack 283 win 0
===============================================================
I am not sure what the hell is happening.
The same problem occurs when trying to connect to ftp server - ftp
commands work fine but when I'm trying to download file and massive
tcp connection forms connection hangs.
I would appriciate any useful information on this topic and
information on how can I debug this more deeply.
--
Best regards,
Oleg Tarasov mailto:subscriber at osk.com.ua
-------------- next part --------------
Copyright (c) 1992-2005 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD 6.0-RELEASE #0: Tue Nov 29 15:32:53 EET 2005
root at gandalf.piramida.com.ua:/usr/obj/usr/src/sys/PIRAMIDA
WARNING: debug.mpsafenet forced to 0 as ipsec requires Giant
WARNING: MPSAFE network stack disabled, expect reduced performance.
Timecounter "i8254" frequency 1193182 Hz quality 0
CPU: Intel(R) Celeron(TM) CPU 1100MHz (1093.90-MHz 686-class CPU)
Origin = "GenuineIntel" Id = 0x6b1 Stepping = 1
Features=0x383f9ff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,MMX,FXSR,SSE>
real memory = 402587648 (383 MB)
avail memory = 384339968 (366 MB)
npx0: [FAST]
npx0: <math processor> on motherboard
npx0: INT 16 interface
acpi0: <IntelR AWRDACPI> on motherboard
acpi0: Power Button (fixed)
pci_link0: <ACPI PCI Link LNKA> irq 9 on acpi0
pci_link1: <ACPI PCI Link LNKB> irq 11 on acpi0
pci_link2: <ACPI PCI Link LNKC> irq 11 on acpi0
pci_link3: <ACPI PCI Link LNKD> irq 5 on acpi0
pci_link4: <ACPI PCI Link LNKE> irq 0 on acpi0
pci_link5: <ACPI PCI Link LNKF> irq 0 on acpi0
pci_link6: <ACPI PCI Link LNK0> irq 0 on acpi0
pci_link7: <ACPI PCI Link LNK1> irq 11 on acpi0
Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0
cpu0: <ACPI CPU> on acpi0
acpi_throttle0: <ACPI CPU Throttling> on cpu0
acpi_button0: <Power Button> on acpi0
acpi_button1: <Sleep Button> on acpi0
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
agp0: <Intel 82815 (i815 GMCH) host to PCI bridge> mem 0xe8000000-0xebffffff at device 0.0 on pci0
pcib1: <PCI-PCI bridge> at device 1.0 on pci0
pci1: <PCI bus> on pcib1
pci1: <display, VGA> at device 0.0 (no driver attached)
pcib2: <ACPI PCI-PCI bridge> at device 30.0 on pci0
pci2: <ACPI PCI bus> on pcib2
rl0: <RealTek 8139 10/100BaseTX> port 0xc000-0xc0ff mem 0xee000000-0xee0000ff irq 11 at device 2.0 on pci2
miibus0: <MII bus> on rl0
rlphy0: <RealTek internal media interface> on miibus0
rlphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
rl0: Ethernet address: 00:30:4f:1c:ed:19
rl0: [GIANT-LOCKED]
rl1: <RealTek 8139 10/100BaseTX> port 0xc400-0xc4ff mem 0xee001000-0xee0010ff irq 5 at device 3.0 on pci2
miibus1: <MII bus> on rl1
rlphy1: <RealTek internal media interface> on miibus1
rlphy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
rl1: Ethernet address: 00:30:4f:1c:ed:17
rl1: [GIANT-LOCKED]
isab0: <PCI-ISA bridge> at device 31.0 on pci0
isa0: <ISA bus> on isab0
atapci0: <Intel ICH2 UDMA100 controller> port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xf000-0xf00f at device 31.1 on pci0
ata0: <ATA channel 0> on atapci0
ata1: <ATA channel 1> on atapci0
uhci0: <Intel 82801BA/BAM (ICH2) USB controller USB-A> port 0xd000-0xd01f irq 5 at device 31.2 on pci0
uhci0: [GIANT-LOCKED]
usb0: <Intel 82801BA/BAM (ICH2) USB controller USB-A> on uhci0
usb0: USB revision 1.0
uhub0: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
uhub0: 2 ports with 2 removable, self powered
pci0: <serial bus, SMBus> at device 31.3 (no driver attached)
uhci1: <Intel 82801BA/BAM (ICH2) USB controller USB-B> port 0xd800-0xd81f irq 11 at device 31.4 on pci0
uhci1: [GIANT-LOCKED]
usb1: <Intel 82801BA/BAM (ICH2) USB controller USB-B> on uhci1
usb1: USB revision 1.0
uhub1: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
uhub1: 2 ports with 2 removable, self powered
pci0: <multimedia, audio> at device 31.5 (no driver attached)
acpi_tz0: <Thermal Zone> on acpi0
fdc0: <floppy drive controller> port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on acpi0
fdc0: [FAST]
fd0: <1440-KB 3.5" drive> on fdc0 drive 0
sio0: <16550A-compatible COM port> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
sio0: type 16550A
sio1: <16550A-compatible COM port> port 0x2f8-0x2ff irq 3 on acpi0
sio1: type 16550A
ppc0: <Standard parallel printer port> port 0x378-0x37f irq 7 on acpi0
ppc0: Generic chipset (NIBBLE-only) in COMPATIBLE mode
ppbus0: <Parallel port bus> on ppc0
plip0: <PLIP network interface> on ppbus0
lpt0: <Printer> on ppbus0
lpt0: Interrupt-driven port
ppi0: <Parallel I/O> on ppbus0
atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
psm0: <PS/2 Mouse> irq 12 on atkbdc0
psm0: [GIANT-LOCKED]
psm0: model NetMouse/NetScroll Optical, device ID 0
pmtimer0 on isa0
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x300>
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
Timecounter "TSC" frequency 1093902442 Hz quality 800
Timecounters tick every 1.000 msec
IPsec: Initialized Security Association Processing.
ipfw2 (+ipv6) initialized, divert loadable, rule-based forwarding enabled, default to accept, logging limited to 300 packets/entry by default
ad0: 38204MB <SAMSUNG SP0411N TW100-08> at ata0-master UDMA100
acd0: CDROM <ASUS CD-S520/A4/1.2> at ata1-master UDMA33
Trying to mount root from ufs:/dev/ad0s1a
rl1: link state changed to UP
-------------- next part --------------
A non-text attachment was scrubbed...
Name: KERNEL
Type: application/octet-stream
Size: 10757 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-net/attachments/20051226/347c28b9/KERNEL.obj
-------------- next part --------------
A non-text attachment was scrubbed...
Name: mpd
Type: application/octet-stream
Size: 775 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-net/attachments/20051226/347c28b9/mpd.obj
-------------- next part --------------
A non-text attachment was scrubbed...
Name: mpd.conf
Type: application/octet-stream
Size: 594 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-net/attachments/20051226/347c28b9/mpd-0001.obj
-------------- next part --------------
A non-text attachment was scrubbed...
Name: mpd.links
Type: application/octet-stream
Size: 154 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-net/attachments/20051226/347c28b9/mpd-0002.obj
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ppp.conf
Type: application/octet-stream
Size: 750 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-net/attachments/20051226/347c28b9/ppp.obj
-------------- next part --------------
A non-text attachment was scrubbed...
Name: rc.firewall
Type: application/octet-stream
Size: 8150 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-net/attachments/20051226/347c28b9/rc.obj
More information about the freebsd-net
mailing list