kern/84802: kernel hangs with pf and route-to
Sergey Lapin
slapinid at gmail.com
Thu Aug 11 14:50:05 GMT 2005
>Number: 84802
>Category: kern
>Synopsis: kernel hangs with pf and route-to
>Confidential: no
>Severity: critical
>Priority: medium
>Responsible: freebsd-bugs
>State: open
>Quarter:
>Keywords:
>Date-Required:
>Class: sw-bug
>Submitter-Id: current-users
>Arrival-Date: Thu Aug 11 14:50:04 GMT 2005
>Closed-Date:
>Last-Modified:
>Originator: Sergey Lapin
>Release: FreeBSD 6.0-BETA2 i386
>Organization:
>Environment:
System: FreeBSD int-s60-3.dart.spb 6.0-BETA2 FreeBSD 6.0-BETA2 #0: Wed Aug 3 08:22:24 UTC 2005 root at harlow.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC i386
>Description:
We are have 2 upstream providers. They give us 2 networks.
First, 1.1.1.0/24 is split by us into 2 parts by /25 mask - first part
is outer, and second (129-254) is behind our router (DMZ1)
second is 2.2.2.0/25 given by second ISP, is routed to our router,
and provider gives /30 p-to-p network 2.2.3.172/30 so 174 is our side and
173 is their's. So 2.2.2.0/25 is routed to 2.2.3.174 from their side.
Configuration:
(all addresses fake, 1.1.1.x - from ISP1, 2.2.2 - from ISP2)
# grep ifconfig /etc/rc.conf
ifconfig_xl0="inet 1.1.1.254 netmask 255.255.255.128"
ifconfig_xl0_alias0="inet 2.2.2.2 netmask 255.255.255.128"
ifconfig_xl1="inet 192.168.255.1 netmask 255.255.255.255"
ifconfig_vlan0="inet 1.1.1.3 netmask 255.255.255.0 vlan 1001 vlandev xl1 mtu 1496"
ifconfig_vlan1="inet 2.2.3.174 netmask 255.255.255.252 vlan 1004 vlandev xl1 mtu 1496"
# grep defaultrouter /etc/rc.conf
defaultrouter="1.1.1.1"
# cat /etc/pf.conf
# $FreeBSD: src/etc/pf.conf,v 1.1.2.1 2004/09/17 18:27:14 mlaier Exp $
# $OpenBSD: pf.conf,v 1.21 2003/09/02 20:38:44 david Exp $
#
# See pf.conf(5) and /usr/share/examples/pf for syntax and examples.
# Required order: options, normalization, queueing, translation, filtering.
# Macros and tables may be defined and used anywhere.
# Note that translation rules are first match while filter rules are last match.
ext_if1 = "vlan0"
ext_if2 = "vlan1"
dmz_if = "xl0"
ext_gw1 = "1.1.1.1"
ext_gw2 = "2.2.3.173"
lan_net = "192.168.0.0/16"
dmz_net1 = "1.1.1.128/25"
dmz_net2 = "2.2.2.0/25"
table <our_nets> const { $dmz_net1, $dmz_net2, $lan_net }
set block-policy drop
set state-policy floating
# Normalize all incoming streams
scrub in on $ext_if1
scrub in on $ext_if2
#################################################################################
# NAT
#
#################################################################################
# nat outgoing connections on each internet interface
nat on $ext_if1 from { $lan_net $dmz_net2 } to any -> ($ext_if1)
nat on $ext_if2 from { $lan_net $dmz_net1 } to any -> ($ext_if2)
#################################################################################
# Block everything by default
#
#################################################################################
# default deny silently
block drop all
# block IDENT notifying sender to prevent sendmail and the like from
# wasting time waiting for timeout
block return in on { $ext_if1 $ext_if2 } proto { tcp, udp } to port = auth
block drop log on xl0 all
#################################################################################
# Traffic to gateway itself
#
#################################################################################
# pass in quick any packets destined for the gateway itself
pass in quick on $dmz_if proto tcp from any to $dmz_if flags S/SA keep state
pass in quick on $dmz_if inet proto { udp, icmp } from any to $dmz_if keep state
# pass multicast and IGMP traffic
pass quick on $dmz_if inet from any to 224.0.0.0/4 allow-opts keep state
pass quick on lo0
#################################################################################
# Classify traffic from DMZ
#
#################################################################################
# pass traffic from DMZ to Internet
pass in on $dmz_if proto udp from $dmz_net1 to any port = 53 keep state tag DMZ_TO_EXT1
pass in on $dmz_if proto udp from $dmz_net2 to any port = 53 keep state tag DMZ_TO_EXT2
# Allow all outgoing connections from DMZ
pass in on $dmz_if inet proto tcp from $dmz_net1 to any flags S/SA keep state tag DMZ_TO_EXT1
pass in on $dmz_if inet proto { udp, icmp } from $dmz_net1 to any keep state tag DMZ_TO_EXT1
pass in on $dmz_if inet proto tcp from $dmz_net2 to any flags S/SA keep state tag DMZ_TO_EXT2
pass in on $dmz_if inet proto { udp, icmp } from $dmz_net2 to any keep state tag DMZ_TO_EXT2
# Allow gateway to route between different networks on the DMZ
# DMZ nets -> DMZ nets
pass in on $dmz_if inet proto tcp from { $dmz_net1, $dmz_net2 } to { $dmz_net1, $dmz_net2 } flags S/SA keep state tag DMZ_TO_DMZ
pass in on $dmz_if inet from { $dmz_net1, $dmz_net2 } to { $dmz_net1, $dmz_net2 } keep state tag DMZ_TO_DMZ
# DMZ nets -> LAN net
pass in on $dmz_if inet proto tcp from { $dmz_net1, $dmz_net2 } to $lan_net flags S/SA keep state tag DMZ_TO_LAN
pass in on $dmz_if inet from { $dmz_net1, $dmz_net2 } to $lan_net keep state tag DMZ_TO_LAN
# LAN net -> DMZ nets
pass in on $dmz_if inet proto tcp from $lan_net to { $dmz_net1, $dmz_net2 } flags S/SA keep state tag LAN_TO_DMZ
pass in on $dmz_if inet from $lan_net to { $dmz_net1, $dmz_net2 } keep state tag LAN_TO_DMZ
#################################################################################
# Allow classified traffic from DMZ
#
#################################################################################
# Allow incoming packets from DMZ one more time and route them appropriately
# This must be done to IN packets because if we only do it for OUT
packets, it happens to late -
# packet is routed appropriately, but NAT rule for wrong interface gets fired
pass in quick on $dmz_if route-to ($ext_if1 $ext_gw1) proto tcp tagged DMZ_TO_EXT1 flags S/SA modulate state
pass in quick on $dmz_if route-to ($ext_if1 $ext_gw1) proto { udp, icmp } tagged DMZ_TO_EXT1 keep state
pass in quick on $dmz_if route-to ($ext_if2 $ext_gw2) proto tcp tagged DMZ_TO_EXT2 flags S/SA modulate state
pass in quick on $dmz_if route-to ($ext_if2 $ext_gw2) proto { udp, icmp } tagged DMZ_TO_EXT2 keep state
# Allow OUT traffic
pass out quick on $ext_if1 route-to ($ext_if2 $ext_gw2) proto tcp tagged DMZ_TO_EXT2 flags S/SA modulate state
pass out quick on $ext_if1 route-to ($ext_if2 $ext_gw2) proto { udp, icmp } tagged DMZ_TO_EXT2 keep state
pass out quick on $ext_if2 route-to ($ext_if1 $ext_gw1) proto tcp tagged DMZ_TO_EXT1 flags S/SA modulate state
pass out quick on $ext_if2 route-to ($ext_if1 $ext_gw1) proto { udp, icmp } tagged DMZ_TO_EXT1 keep state
#################################################################################
# Clasify traffic from Internet to DMZ
#
#################################################################################
# WHISKEY
pass in on vlan0 proto tcp from any to 1.1.1.144/32 port = 22 flags S/SA keep state tag EXT1_TO_DMZ
pass in on vlan1 proto tcp from any to 2.2.2.2/32 port = 22 flags S/SA keep state tag EXT2_TO_DMZ
#################################################################################
# Allow classified traffic from Internet to DMZ
#
#################################################################################
# Pass to DMZ traffic already approved by earlier rules
# and route replies to corresponding interface
# EXT1
pass out quick on $dmz_if reply-to ($ext_if1 $ext_gw1) proto tcp tagged EXT1_TO_DMZ flags S/SA keep state
pass out quick on $dmz_if reply-to ($ext_if1 $ext_gw1) tagged EXT1_TO_DMZ keep state
# EXT2
pass out quick on $dmz_if reply-to ($ext_if2 $ext_gw2) proto tcp tagged EXT2_TO_DMZ flags S/SA keep state
pass out quick on $dmz_if reply-to ($ext_if2 $ext_gw2) tagged EXT2_TO_DMZ keep state
#################################################################################
# Other traffic
#
#################################################################################
# general "pass out" rules for external interfaces
pass out on { $ext_if1, $ext_if2, $dmz_if } proto tcp from any to any flags S/SA modulate state
pass out on { $ext_if1, $ext_if2, $dmz_if } proto { udp, icmp } from any to any keep state
# Zebra uses IGMP so let it work on DMZ interface
pass out on $dmz_if proto igmp from any to any allow-opts
Test case:
(done from Linix machine from 1.1.1.128/25)
tcpreplay -e 1.1.1.133:255.255.255.255 -i eth0 packet
(where packet is random captured _broadcast_ UDP packet using tcpdump -peni)
or
tcpreplay -e 1.1.1.133:10.2.2.2 -i eth0 packet
(where packet is random captured _broadcast_ UDP packet)
kills machine.
Machine hangs and doesn't react on keyboard, whatever.
Only reset helps.
More than that - machine sends that packet throught interface (that the same packet) continuously, like in loop, even while we're not sending it.
I don't need to say that the same ruleset works well on OpenBSD 3.7 (with s/modulate/keep/g)
Any ideas?
Thanks a lot!
>How-To-Repeat:
Just read description, find UDP packet with broadcast mac and try to sen
d it.
It doesn't reproduce inside VmWare.
>Fix:
Temp. workaround idea is disable broadcast UDP packets, but I don't
think it's too good.
>Release-Note:
>Audit-Trail:
>Unformatted:
More information about the freebsd-bugs
mailing list