very strange problem
Matt Douhan
mdouhan at fruitsalad.org
Sat Jul 12 12:53:08 PDT 2003
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Sorry for topposting but I will try and answer the requests one by one, I can
only do FW1 today, and fw2 on monday, but here goes
>
> possible send tcpump record pb ?
> (example: tcpdump -ns 0 -i externalintf_fw1 -w all1.tcpdump
> and tcpdump -ns 0 -i externalintf_fw2 -w all2.tcpdump)
dump is pretty large so I did not want to email it, please download it from
http://www.fruitsalad.org/people/mdouhan/fw1.tar.gz
>
> possible send ipf -V (on two fw) ?
7:47pm mdouhan @ [firewall1] ~ > sudo ipf -V
ipf: IP Filter: v3.4.31 (336)
Kernel: IP Filter: v3.4.31
Running: yes
Log Flags: 0 = none set
Default: pass all, Logging: available
Active list: 0
>
> possible send ipfstat -nhio (on two fw) ?
>
7:49pm mdouhan @ [firewall1] ~ > sudo ipfstat -nhio
2073551 @1 pass out quick on fxp0 from any to any keep state
1038 @1 pass in quick on fxp0 proto icmp from any to any
1802016 @2 pass in quick on fxp0 from 192.168.254.242/32 to 192.168.15.250/32
1255 @3 pass in quick on fxp0 from 192.168.254.250/32 to 192.168.15.249/32
372304 @4 block in log quick on fxp0 from any to any
> possible send ipnat -slv (on two fw) ?
fw1 is not running NAT, will sedn this on monday when I get to fw2
>
> possible send netstat -ni ?
>
7:50pm mdouhan @ [firewall1] ~ > netstat -ni
Name Mtu Network Address Ipkts Ierrs Opkts Oerrs
Coll
fxp0 1500 <Link#1> 00:02:b3:cc:20:6e 45474907 0 46776572 0
0
fxp0 1500 192.168.254 192.168.254.1 612 - 673 -
- -
fxp0 1500 fe80:1::202:b fe80:1::202:b3ff: 0 - 0 -
- -
fxp1 1500 <Link#2> 00:02:b3:cc:1b:3f 47307566 3 45127446 0
0
fxp1 1500 192.168.15 192.168.15.254 184152 - 40018 -
- -
fxp1 1500 fe80:2::202:b fe80:2::202:b3ff: 0 - 0 -
- -
lp0* 1500 <Link#3> 0 0 0 0
0
lo0 16384 <Link#4> 528 0 528 0
0
lo0 16384 ::1/128 ::1 0 - 0 -
- -
lo0 16384 fe80:4::1/64 fe80:4::1 0 - 0 -
- -
lo0 16384 127 127.0.0.1 528 - 528 -
- -
> possible send ifconfig -a ?
>
7:50pm mdouhan @ [firewall1] ~ > ifconfig -a
fxp0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
options=3<RXCSUM,TXCSUM>
inet 192.168.254.1 netmask 0xffffff00 broadcast 192.168.254.255
inet6 fe80::202:b3ff:fecc:206e%fxp0 prefixlen 64 scopeid 0x1
ether 00:02:b3:cc:20:6e
media: Ethernet autoselect (100baseTX <full-duplex>)
status: active
fxp1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
options=3<RXCSUM,TXCSUM>
inet 192.168.15.254 netmask 0xffffff00 broadcast 192.168.15.255
inet6 fe80::202:b3ff:fecc:1b3f%fxp1 prefixlen 64 scopeid 0x2
ether 00:02:b3:cc:1b:3f
media: Ethernet autoselect (100baseTX <full-duplex>)
status: active
lp0: flags=8810<POINTOPOINT,SIMPLEX,MULTICAST> mtu 1500
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 16384
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x4
inet 127.0.0.1 netmask 0xff000000
> possible dmesg ?
>
7:51pm mdouhan @ [firewall1] ~ > dmesg
Copyright (c) 1992-2003 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD 5.1-CURRENT #2: Wed Jul 2 15:40:03 GMT 2003
root at firewall1.internal.hasta.se:/usr/obj/usr/src/sys/FIREWALL1
Preloaded elf kernel "/boot/kernel/kernel" at 0xc052a000.
Preloaded elf module "/boot/kernel/acpi.ko" at 0xc052a1cc.
Timecounter "i8254" frequency 1193182 Hz
Timecounter "TSC" frequency 1799806528 Hz
CPU: Intel(R) Celeron(R) CPU 1.80GHz (1799.81-MHz 686-class CPU)
Origin = "GenuineIntel" Id = 0xf13 Stepping = 3
Features=0x3febfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM>
real memory = 536805376 (511 MB)
avail memory = 515776512 (491 MB)
Pentium Pro MTRR support enabled
npx0: <math processor> on motherboard
npx0: INT 16 interface
acpi0: <AOpen AWRDACPI> on motherboard
pcibios: BIOS version 2.10
Using $PIR table, 11 entries at 0xc00fdeb0
acpi0: power button is handled as a fixed feature programming model.
Timecounter "ACPI-fast" frequency 3579545 Hz
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x4008-0x400b on acpi0
acpi_cpu0: <CPU> on acpi0
acpi_cpu1: <CPU> on acpi0
acpi_tz0: <thermal zone> on acpi0
acpi_button0: <Power Button> on acpi0
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
pcib0: slot 29 INTA is routed to irq 12
pcib0: slot 29 INTB is routed to irq 11
pcib0: slot 29 INTC is routed to irq 12
pcib0: slot 29 INTD is routed to irq 10
pcib0: slot 31 INTB is routed to irq 11
pcib0: slot 31 INTB is routed to irq 11
agp0: <Intel 82845 host to AGP bridge> mem 0xe0000000-0xe3ffffff at device 0.0
on pci0
pcib1: <PCI-PCI bridge> at device 1.0 on pci0
pci1: <PCI bus> on pcib1
pcib0: slot 1 INTA is routed to irq 12
pcib1: slot 0 INTA is routed to irq 12
pci1: <display, VGA> at device 0.0 (no driver attached)
uhci0: <Intel 82801DB (ICH4) USB controller USB-A> port 0xd800-0xd81f irq 12
at device 29.0 on pci0
usb0: <Intel 82801DB (ICH4) USB controller USB-A> on uhci0
usb0: USB revision 1.0
uhub0: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
uhub0: 2 ports with 2 removable, self powered
uhci1: <Intel 82801DB (ICH4) USB controller USB-B> port 0xd000-0xd01f irq 11
at device 29.1 on pci0
usb1: <Intel 82801DB (ICH4) USB controller USB-B> on uhci1
usb1: USB revision 1.0
uhub1: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
uhub1: 2 ports with 2 removable, self powered
uhci2: <Intel 82801DB (ICH4) USB controller USB-C> port 0xd400-0xd41f irq 12
at device 29.2 on pci0
usb2: <Intel 82801DB (ICH4) USB controller USB-C> on uhci2
usb2: USB revision 1.0
uhub2: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
uhub2: 2 ports with 2 removable, self powered
pci0: <serial bus, USB> at device 29.7 (no driver attached)
pcib2: <ACPI PCI-PCI bridge> at device 30.0 on pci0
pci2: <ACPI PCI bus> on pcib2
pcib2: slot 7 INTA is routed to irq 11
pcib2: slot 9 INTA is routed to irq 10
fxp0: <Intel 82557/8/9 EtherExpress Pro/100(B) Ethernet> port 0xc000-0xc03f
mem 0xe9000000-0xe901ffff,0xe9041000-0xe9041fff irq 11 at device 7.0 on pci2
fxp0: Ethernet address 00:02:b3:cc:20:6e
miibus0: <MII bus> on fxp0
inphy0: <i82555 10/100 media interface> on miibus0
inphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
fxp1: <Intel 82557/8/9 EtherExpress Pro/100(B) Ethernet> port 0xc400-0xc43f
mem 0xe9020000-0xe903ffff,0xe9040000-0xe9040fff irq 10 at device 9.0 on pci2
fxp1: Ethernet address 00:02:b3:cc:1b:3f
miibus1: <MII bus> on fxp1
inphy1: <i82555 10/100 media interface> on miibus1
inphy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
isab0: <PCI-ISA bridge> at device 31.0 on pci0
isa0: <ISA bus> on isab0
atapci0: <Intel ICH4 UDMA100 controller> port
0xf000-0xf00f,0-0x3,0-0x7,0-0x3,0-0x7 at device 31.1 on pci0
ata0: at 0x1f0 irq 14 on atapci0
ata1: at 0x170 irq 15 on atapci0
pci0: <serial bus, SMBus> at device 31.3 (no driver attached)
pci0: <multimedia, audio> at device 31.5 (no driver attached)
fdc0: <Enhanced floppy controller (i82077, NE72065 or clone)> port
0x3f7,0x3f0-0x3f5 irq 6 drq 2 on acpi0
fdc0: FIFO enabled, 8 bytes threshold
fd0: <1440-KB 3.5" drive> on fdc0 drive 0
sio0 port 0x3f8-0x3ff irq 4 on acpi0
sio0: type 16550A
sio1 port 0x2f8-0x2ff irq 3 on acpi0
sio1: type 16550A
ppc0 port 0x778-0x77b,0x378-0x37f irq 7 on acpi0
ppc0: Generic chipset (NIBBLE-only) in COMPATIBLE mode
ppbus0: <Parallel port bus> on ppc0
plip0: <PLIP network interface> on ppbus0
lpt0: <Printer> on ppbus0
lpt0: Interrupt-driven port
ppi0: <Parallel I/O> on ppbus0
orm0: <Option ROMs> at iomem 0xce000-0xcf7ff,0xcc000-0xcd7ff,0xc0000-0xca7ff
on isa0
pmtimer0 on isa0
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x300>
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
Timecounters tick every 10.000 msec
IP Filter: v3.4.31 initialized. Default = pass all, Logging = enabled
acpi_cpu: throttling enabled, 2 steps (100% to 50.0%), currently 100.0%
ata1-master: timeout waiting for interrupt
ata1-master: ATAPI identify failed
ad0: 38166MB <WDC WD400BB-00DEA0> [77545/16/63] at ata0-master UDMA100
Mounting root from ufs:/dev/ad0s1a
IP Filter: already initialized
IP Filter: already initialized
fxp0: promiscuous mode enabled
fxp0: promiscuous mode disabled
fxp0: promiscuous mode enabled
fxp0: promiscuous mode disabled
fxp0: promiscuous mode enabled
fxp0: promiscuous mode disabled
7:51pm mdouhan @ [firewall1] ~ >
> Regard.
>
> Matt Douhan wrote:
> > -----BEGIN PGP SIGNED MESSAGE-----
> > Hash: SHA1
> >
> > Hello
> >
> > I am running FBSD on two firewalls in a scenario like below
> >
> > internet
> >
> > FW2
> >
> > DMZ
> >
> > FW1
> >
> > internal LAN
> >
> > FW1 is running ipf and fw2 is running ipf and ipnat
> >
> > hosts on the DMZ can access the internet without problems, ping
> > traceroute and mail, http all is working nicely and fast.
> >
> > hosts on the internal LAN however are seing VERY strange things
> >
> > for example, check this out
> >
> > 9:04pm mdouhan @ [persika] ~ > traceroute www.cisco.com
> > traceroute to www.cisco.com (198.133.219.25), 64 hops max, 40 byte
> > packets 1 192.168.15.254 (192.168.15.254) 0.698 ms 0.532 ms 0.410 ms
> > 2 192.168.254.254 (192.168.254.254) 0.781 ms 0.757 ms 0.744 ms 3
> > gw-l3-ktv-hc.koping.net (81.16.160.113) 1.210 ms 1.203 ms 1.263 ms 4
> > gw-l3-ktv-it.koping.net (81.16.160.6) 1.546 ms 4.123 ms 1.272 ms 5
> > rif3-r1-jvg-kop.arrowhead.com (81.216.90.1) 3.336 ms 2.813 ms 2.649 ms
> > 6 www.cisco.com (198.133.219.25) 1.278 ms 2.610 ms 1.962 ms
> >
> > the host "persika" is connected on the internal LAN, and is located in
> > Sweden, Europe and there is NO way it can get to www.cisco.com in 2-3 ms,
> > and I dont have any caching or proxies or anything, besides traceroute
> > does not care about that anyway AFAIK
> >
> > same traceroute from a host on the DMZ shows the correct thing as follows
> >
> > 9:05pm mdouhan @ [ananas] ~ > traceroute www.cisco.com
> > traceroute to www.cisco.com (198.133.219.25), 64 hops max, 40 byte
> > packets 1 firewall2 (192.168.254.254) 0.671 ms 0.458 ms 0.438 ms
> > 2 gw-l3-ktv-hc.koping.net (81.16.160.113) 0.901 ms 0.931 ms 0.878 ms
> > 3 gw-l3-ktv-it.koping.net (81.16.160.6) 1.416 ms 1.191 ms 1.388 ms
> > 4 rif3-r1-jvg-kop.arrowhead.com (81.216.90.1) 2.345 ms 2.080 ms
> > 2.705 ms 5 rif2-cr1-vf-kop.arrowhead.com (81.216.2.1) 1.973 ms 2.173
> > ms 2.263 ms 6 rif6-cr1-vf-vst.arrowhead.com (81.216.0.53) 3.785 ms
> > 2.708 ms 2.540 ms 7 rif3-cr1-vf-oby.arrowhead.com (213.187.195.97)
> > 3.363 ms 16.022 ms 3.862 ms
> > 8 rif47-rs1-t4-sto.arrowhead.com (213.187.195.93) 4.769 ms 4.396 ms
> > 3.999 ms
> > 9 rif5-cr3-kst-sto.arrowhead.com (81.216.0.137) 5.115 ms 4.624 ms
> > 4.762 ms
> > 10 Gi14-1-kst-p1.sto.se.sn.net (81.216.0.113) 4.496 ms 4.577 ms 4.666
> > ms 11 pos2-0.vrt-p1.sto.se.sn.net (213.88.255.245) 4.687 ms 4.757 ms
> > 4.806 ms 12 sl-gw20-sto-2-1.sprintlink.net (80.77.97.89) 4.575 ms
> > 4.526 ms 4.576 ms 13 sl-bb21-sto-12-0.sprintlink.net (80.77.96.98)
> > 4.969 ms 5.132 ms 5.526 ms
> > 14 sl-bb21-cop-12-0.sprintlink.net (213.206.129.33) 14.034 ms * 13.904
> > ms 15 sl-bb20-cop-15-0.sprintlink.net (80.77.64.33) 13.942 ms 13.498
> > ms 13.966 ms
> > 16 sl-bb21-msq-10-0.sprintlink.net (144.232.19.29) 91.125 ms 102.015
> > ms 93.908 ms
> > 17 sl-bb22-rly-15-3.sprintlink.net (144.232.19.98) 96.692 ms 95.680 ms
> > 96.615 ms
> > 18 sl-bb25-rly-12-0.sprintlink.net (144.232.14.166) 96.692 ms 95.879
> > ms 95.900 ms
> > 19 sl-bb23-sj-9-0.sprintlink.net (144.232.20.11) 227.115 ms 241.136 ms
> > 220.680 ms
> > 20 sl-bb25-sj-14-0.sprintlink.net (144.232.3.250) 181.269 ms 173.322
> > ms 164.253 ms
> > 21 sl-gw11-sj-10-0.sprintlink.net (144.232.3.134) 172.763 ms 172.362
> > ms 172.324 ms
> > 22 sl-ciscopsn2-11-0-0.sprintlink.net (144.228.44.14) 166.180 ms
> > 166.028 ms 170.228 ms
> > 23 sjck-dirty-gw1.cisco.com (128.107.239.5) 164.721 ms 166.063 ms
> > 166.174 ms
> > 24 sjck-sdf-ciod-gw2.cisco.com (128.107.239.110) 172.908 ms 173.340 ms
> > 173.284 ms
> > 25 www.cisco.com (198.133.219.25) 174.149 ms 174.768 ms *
> >
> > now here is where it gets really weird, I have tries reinstalling FW1
> > since it seems to be the cause of the problem, I have tries STABLE,
> > CURRENT, 5.1-R all with the same result, it does NOT work.
> >
> > I have tried swapping FW1 and FW2 and the problem stays the same, so it
> > seems to be a misconfiguration on my part (or a bug but thats less likely
> > I think) but I cannot figure out what it is.
> >
> > my rules are very simple
> >
> > on FW1 allow anything out on the external fxp interface with keep state
> > so it can get back in.
> >
> > on FW2 I have a number of BIMAP statements and some NAT statements, BIMAP
> > are for the servers where we provide services such as mail, www and ftp.
> >
> > Any input or ideas would be highly appreciated, this is driving me crazy
> >
> > - --
> > -
> > -------------------------------------------------------------------------
> >----------- Matt Douhan
> > www.fruitsalad.org
> > CCIE #4004
> > *** ping elvis ***
> > *** elvis is alive ***
> > -----BEGIN PGP SIGNATURE-----
> > Version: GnuPG v1.2.2 (FreeBSD)
> >
> > iD8DBQE/EF0skU5PITZniCURArKOAJ9HuNWbWCJiV0PRMSpFCo5bv4P3aACfXhAn
> > 9G8PqZQeZZ8RUIABr12VA5Q=
> > =Kda6
> > -----END PGP SIGNATURE-----
> >
> > _______________________________________________
> > freebsd-net at freebsd.org mailing list
> > http://lists.freebsd.org/mailman/listinfo/freebsd-net
> > To unsubscribe, send any mail to "freebsd-net-unsubscribe at freebsd.org"
- --
- ------------------------------------------------------------------------------------
Matt Douhan
www.fruitsalad.org
CCIE #4004
*** ping elvis ***
*** elvis is alive ***
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.2 (FreeBSD)
iD8DBQE/EGcskU5PITZniCURAloQAKC24SRdbrYOM6a1oCEM9nLBiQEmfACfcrVM
Y0jjV2L902CxGFgjkZ/Uoeo=
=HE41
-----END PGP SIGNATURE-----
More information about the freebsd-net
mailing list