i386/80572: bridge/ipfw works intermittantly.

[Josef Karthauser]

>Number:         80572
>Category:       i386
>Synopsis:       bridge/ipfw works intermittantly.
>Confidential:   no
>Severity:       critical
>Priority:       high
>Responsible:    freebsd-i386
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Tue May 03 07:30:01 GMT 2005
>Closed-Date:
>Last-Modified:
>Originator:     Josef Karthauser
>Release:        FreeBSD 5.4-RC4 i386
>Organization:
>Environment:
System: FreeBSD transwarp.tao.org.uk 5.4-RC4 FreeBSD 5.4-RC4 #44: Mon May 2 21:10:57 BST 2005 joe at transwarp.tao.org.uk:/usr/obj/usr/src/sys/TRANSWARP i386

	This machine has just been upgraded from 4.11 to 5.4-rc3
	using a buildworld.

	The box is an intel SMP (dual processor PIII-800MHz) with
	two fxp ethernet interfaces.

Copyright (c) 1992-2005 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
	The Regents of the University of California. All rights reserved.
FreeBSD 5.4-RC4 #44: Mon May  2 21:10:57 BST 2005
    joe at transwarp.tao.org.uk:/usr/obj/usr/src/sys/TRANSWARP
ACPI APIC Table: <Intel  N440BX  >
Timecounter "i8254" frequency 1193182 Hz quality 0
CPU: Intel Pentium III (796.54-MHz 686-class CPU)
  Origin = "GenuineIntel"  Id = 0x683  Stepping = 3
  Features=0x383fbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,MMX,FXSR,SSE>
real memory  = 536805376 (511 MB)
avail memory = 515629056 (491 MB)
ioapic0 <Version 1.1> irqs 0-23 on motherboard
npx0: <math processor> on motherboard
npx0: INT 16 interface
acpi0: <PTLTD   RSDT> on motherboard
acpi0: Power Button (fixed)
Timecounter "ACPI-safe" frequency 3579545 Hz quality 1000
acpi_timer0: <24-bit timer at 3.579545MHz> port 0xc08-0xc0b on acpi0
cpu0: <ACPI CPU> on acpi0
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
agp0: <Intel 82443GX host to PCI bridge> mem 0xf8000000-0xfbffffff at device 0.0 on pci0
pcib1: <PCI-PCI bridge> at device 1.0 on pci0
pci1: <PCI bus> on pcib1
pcib2: <PCI-PCI bridge> at device 15.0 on pci1
pci2: <PCI bus> on pcib2
ahc0: <Adaptec 2940 Ultra2 SCSI adapter> port 0x3000-0x30ff mem 0xf4300000-0xf4300fff irq 20 at device 4.0 on pci2
aic7890/91: Ultra2 Wide Channel A, SCSI Id=7, 32/253 SCBs
fxp0: <Intel 82559 Pro/100 Ethernet> port 0x3400-0x343f mem 0xf4200000-0xf42fffff,0xf4301000-0xf4301fff irq 23 at device 7.0 on pci2
miibus0: <MII bus> on fxp0
inphy0: <i82555 10/100 media interface> on miibus0
inphy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
fxp0: Ethernet address: 00:d0:b7:74:51:af
ahc1: <Adaptec aic7896/97 Ultra2 SCSI adapter> port 0x2000-0x20ff mem 0xf4100000-0xf4100fff irq 19 at device 12.0 on pci0
aic7896/97: Ultra2 Wide Channel A, SCSI Id=7, 32/253 SCBs
ahc2: <Adaptec aic7896/97 Ultra2 SCSI adapter> port 0x2400-0x24ff mem 0xf4101000-0xf4101fff irq 19 at device 12.1 on pci0
aic7896/97: Ultra2 Wide Channel B, SCSI Id=7, 32/253 SCBs
fxp1: <Intel 82559 Pro/100 Ethernet> port 0x2800-0x283f mem 0xf4000000-0xf40fffff,0xf4102000-0xf4102fff irq 21 at device 14.0 on pci0
miibus1: <MII bus> on fxp1
inphy1: <i82555 10/100 media interface> on miibus1
inphy1:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
fxp1: Ethernet address: 00:d0:b7:88:c8:20
isab0: <PCI-ISA bridge> at device 18.0 on pci0
isa0: <ISA bus> on isab0
atapci0: <Intel PIIX4 UDMA33 controller> port 0x2860-0x286f,0x376,0x170-0x177,0x3f6,0x1f0-0x1f7 at device 18.1 on pci0
ata0: channel #0 on atapci0
ata1: channel #1 on atapci0
uhci0: <Intel 82371AB/EB (PIIX4) USB controller> port 0x2840-0x285f irq 21 at device 18.2 on pci0
usb0: <Intel 82371AB/EB (PIIX4) USB controller> on uhci0
usb0: USB revision 1.0
uhub0: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
uhub0: 2 ports with 2 removable, self powered
pci0: <bridge> at device 18.3 (no driver attached)
pci0: <display, VGA> at device 20.0 (no driver attached)
atkbdc0: <Keyboard controller (i8042)> port 0x64,0x60 irq 1 on acpi0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
kbd0 at atkbd0
fdc0: <floppy drive controller> port 0x3f2-0x3f5 irq 6 drq 2 on acpi0
fd0: <1440-KB 3.5" drive> on fdc0 drive 0
ppc0: <ECP parallel printer port> port 0x778-0x77f,0x378-0x37f irq 7 drq 3 on acpi0
ppc0: Generic chipset (ECP/PS2/NIBBLE) in COMPATIBLE mode
ppc0: FIFO with 16/16/8 bytes threshold
ppbus0: <Parallel port bus> on ppc0
plip0: <PLIP network interface> on ppbus0
lpt0: <Printer> on ppbus0
lpt0: Interrupt-driven port
ppi0: <Parallel I/O> on ppbus0
sio0: <16550A-compatible COM port> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
sio0: type 16550A
sio1: <16550A-compatible COM port> port 0x2f8-0x2ff irq 3 on acpi0
sio1: type 16550A
pmtimer0 on isa0
orm0: <ISA Option ROMs> at iomem 0xcf000-0xcffff,0xce800-0xcefff,0xcd800-0xce7ff,0xc8000-0xcd7ff,0xc0000-0xc7fff on isa0
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x300>
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
aue0: USBs USB 10/100 Fast Ethernet, rev 1.10/1.01, addr 2
miibus2: <MII bus> on aue0
ukphy0: <Generic IEEE 802.3u media interface> on miibus2
ukphy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
aue0: Ethernet address: 00:50:ba:82:4c:24
aue0: if_start running deferred for Giant
Timecounter "TSC" frequency 796540873 Hz quality 800
Timecounters tick every 10.000 msec
ipfw2 initialized, divert disabled, rule-based forwarding disabled, default to deny, logging disabled
ad0: 117246MB <Maxtor 6Y120P0/YAR41BW0> [238216/16/63] at ata0-master UDMA33
acd0: CDROM <TOSHIBA CD-ROM XM-6702B/1007> at ata1-master PIO4
Waiting 2 seconds for SCSI devices to settle
sa0 at ahc0 bus 0 target 5 lun 0
sa0: <Quantum DLT4000 D996> Removable Sequential Access SCSI-2 device 
sa0: 10.000MB/s transfers (10.000MHz, offset 15)
da0 at ahc1 bus 0 target 0 lun 0
da0: <SEAGATE ST336607LC 0006> Fixed Direct Access SCSI-3 device 
da0: 80.000MB/s transfers (40.000MHz, offset 63, 16bit), Tagged Queueing Enabled
da0: 35003MB (71687372 512 byte sectors: 255H 63S/T 4462C)
da1 at ahc1 bus 0 target 1 lun 0
da1: <SEAGATE ST39236LC 0010> Fixed Direct Access SCSI-3 device 
da1: 80.000MB/s transfers (40.000MHz, offset 31, 16bit), Tagged Queueing Enabled
da1: 8761MB (17942584 512 byte sectors: 255H 63S/T 1116C)
da2 at ahc1 bus 0 target 3 lun 0
da2: <IBM DRHS36D 0110> Fixed Direct Access SCSI-3 device 
da2: 80.000MB/s transfers (40.000MHz, offset 15, 16bit), Tagged Queueing Enabled
da2: 35239MB (72170879 512 byte sectors: 255H 63S/T 4492C)
Mounting root from ufs:/dev/da0s1a

>Description:
	Using bridge.ko in conjunction with ipfw.ko to create a
	bridging firewall gives a firewall that works for a few
	minutes and then stops forwarding packets.

	The same hardware was working correctly in this configuration
	under FreeBSD-4.11 until it was upgraded directly to
	RELENG_5_4 (5.4-rc3) yesterday using the 'make world' method.

>How-To-Repeat:
	The bridge is created using:

	/etc/sysctl.conf:

		net.link.ether.bridge.enable=1
		net.link.ether.bridge.ipfw=1
		net.link.ether.bridge.config=fxp0,fxp1

	/etc/rc.conf:
		firewall_enable="YES"           # Set to YES to enable firewall functionality
		firewall_script="/etc/rc.firewall-ours" # Which script to run to set up the fire
wall
		firewall_type="ours"            # Firewall type (see /etc/rc.firewall)

	The rc.firewall-ours file is a modified rc.firewall.  It's actual
	contents aren't important for this problem.

>Fix:

	After a reboot the firewall behaves correctly for a few minutes and
	then stops forwarding packets.  The problem can be fixed by running:

	# kldunload ipfw && sleep 20 && kldload ipfw && /etc/netstart
>Release-Note:
>Audit-Trail:
>Unformatted: