FreeBSD7+ipfw+Vlan

Fri Apr 25 12:42:44 UTC 2008

Hello.  
I use for router:  
Dlink DES-3016 + intel Pro/1000XT + Pentium4 + FreeBSD  
# uname -r  
7.0-RC1  

I use:  
6.2-RELEASE-p11 for my vpn-server and this router with kernel option  
if_bridge. In that time I have 5 NIC's, and my router was switch with  
shaper. But one month ago my VPN-server began hang up. Befor hang up I  
recive by squid message:  

Socket Failure  

The system returned:  
    (24) Too many open files  

AND when I try to reboot or write whatever freeBSD couldn't write letter  
and nothing more.  

In my VPN-server I use ipfw + dummynet too.  

After this I decide do router from my bridge with FreeBSD.  
I rebuild kernel. I after that my VPN-server has uptime ten days (before  
less then one day). But my router began hang up.  
Before this problem's I use Dlink DES-2108 as swtitch more than 1 year.  

#cat /etc/rc.conf  
ifconfig_em0="inet 172.168.1.1  netmask 255.255.255.0"  
ifconfig_vr0="inet 10.11.25.13 netmask 255.255.0.0"  
defaultrouter="10.11.25.1"  
cloned_interfaces="vlan1 vlan2 vlan3 vlan4 vlan5 vlan6 vlan7 vlan8 vlan9  
vlan10"  
ifconfig_vlan1="inet 10.12.1.1 netmask 255.255.255.0 vlan 3 vlandev em0"  
ifconfig_vlan2="inet 10.13.1.1 netmask 255.255.255.0 vlan 4 vlandev em0"  
ifconfig_vlan3="inet 10.14.1.1 netmask 255.255.255.0 vlan 5 vlandev em0"  
ifconfig_vlan4="inet 10.15.1.1 netmask 255.255.255.0 vlan 6 vlandev em0"  
gateway_enable="YES"  
rpcbind_enable="NO"  
ipfw_enable="YES"  
ipfw_enable="YES"  
ipfw_type="OPEN"  
pf_enable="YES"  
pf_rules="/etc/pf.conf"  
router_enable="NO"  
#########dhcp#################  
dhcpd_enable="YES"  
dhcpd_flags="-q"  
dhcpd_ifaces="vlan1 vlan2 vlan3 vlan4"  
dhcpd_chroot_enable="YES"  
dhcpd_conf="/usr/local/etc/dhcpd.conf"    
dhcpd_devfs_enable="YES"  
dhcpd_jail_enable="NO"  

# cat /etc/sysctl.conf  
kern.maxfiles=128000  
kern.maxfilesperproc=65000  
kern.ipc.somaxconn=32768  
net.inet.ip.intr_queue_maxlen=200  
kern.ipc.maxsockbuf=1048576  
net.inet.tcp.sendspace=65535  
net.inet.tcp.recvspace=32768  
net.inet.udp.recvspace=655350  
net.inet.icmp.drop_redirect=1  
net.inet.udp.blackhole=2  
net.inet.tcp.blackhole=2  
net.inet.tcp.msl=7500  
kern.ipc.maxsockets=204800  

# cat /etc/pf.conf  
scrub in all  
pass in all  
pass out all  

#pftop  
pfTop: Up State 1-30/578, View: default, Order: none, Cache: 10000  
14:18:08  

# pfctl -s info  
Status: Enabled for 0 days 00:27:07           Debug: Urgent  

State Table                          Total             Rate  
  current entries                      566  
  searches                         8512194         5231.8/s  
  inserts                            21525           13.2/s  
  removals                           20959           12.9/s  
Counters  
  match                            4340001         2667.5/s  
  bad-offset                             0            0.0/s  
  fragment                               0            0.0/s  
  short                                  0            0.0/s  
  normalize                              0            0.0/s  
  memory                                 0            0.0/s  
  bad-timestamp                          0            0.0/s  
  congestion                             0            0.0/s  
  ip-option                              0            0.0/s  
  proto-cksum                            1            0.0/s  
  state-mismatch                        31            0.0/s  
  state-insert                           0            0.0/s  
  state-limit                            0            0.0/s  
  src-limit                              0            0.0/s  
  synproxy                               0            0.0/s  

#ipfw show  
00008 13848862  8065556536 allow gre from any to any  
00009        0           0 allow udp from any to any dst-port 500  
00010    17332     1051156 allow tcp from any to any dst-port 1023,1723  
00011        0           0 allow esp from any to any  
00024        0           0 allow udp from 0.0.0.0 2054 to 0.0.0.0  
00025        0           0 deny icmp from any to any in icmptypes  
5,9,13,14,15,16,17  
00026        0           0 deny tcp from any to me in tcpflags syn,fin,!  
ack  
00027        0           0 deny tcp from any to me in tcpflags syn,fin,!  
ack,psh,urg  
00028        0           0 deny tcp from any to me in tcpflags fin,!  
ack,psh,urg  
00203     4263      581066 pipe 12 ip from 10.11.25.1 to any via vlan1  
00204     2763      147041 pipe 12 ip from any to 10.11.25.1 via vlan1  
00205  5944333  5438517982 pipe 13 ip from any to any via vlan1  

00206      1585      240264 pipe 14 ip from 10.11.25.1 to any via vlan2  
00207     859       52217 pipe 14 ip from any to 10.11.25.1 via vlan2  
00208  19187     5468180 pipe 15 ip from any to any via vlan2  

00209     0      0 pipe 16 ip from 10.11.25.1 to any via vlan3  
00210     0      0 pipe 16 ip from any to 10.11.25.1 via vlan3  
00211  0  0 pipe 17 ip from any to any via vlan3  

[root at f7RC1 /usr/src/sys/i386/conf]# cat ROUTER  

cpu             I686_CPU  
ident           ROUTER  
options         SCHED_ULE  
options IPFIREWALL  
options IPFIREWALL_VERBOSE  
#options IPDIVERT  
options IPFIREWALL_FORWARD  
#options IPV6FIREWALL  
#options IPV6FIREWALL_VERBOSE  

options DUMMYNET  

options DEVICE_POLLING  

I create Vlan's on DES-3016, with differents VID:  

DES-3016:4#show vlan  
Command: show vlan  
....  
VID             : 3          VLAN Name       : 3  
VLAN Type       : static  
Member ports    : 1,7  
Static ports    : 1,7  
Tagged ports    : 1  
Untagged ports  : 7  

VID             : 4          VLAN Name       : 4  
VLAN Type       : static  
Member ports    : 1,8  
Static ports    : 1,8  
Tagged ports    : 1  
Untagged ports  : 8  

VID             : 5          VLAN Name       : 5  
VLAN Type       : static  
Member ports    : 1,9  
Static ports    : 1,9  
Tagged ports    : 1  
Untagged ports  : 9  

............  

Total Entries  : 10  