Per Protocol Traffic Accounting

[Tyler]

Hi All,

I'm trying to count both in and out traffic on a per-protocol basis. 
With the ultimate goal of MRTG'ing and recording the amount of traffic
in and out of the server for each service.  So far, my Googling, and
mailing list archive searches have not produced a solution.  

What happens is the "pass in" line increments, but the "pass out" does
not.  Probably because I'm keeping state on the inbound connections, so
it thinks that entire flow is on that one rule?  Can that be changed?

My network is fairly simple.  One /24 of 192.168.0.x IP's behind my
firewall/server.  Running typical HTTP, POP3, DNS, etc services on the
firewall/server for both external and internal clients.  The
firewall/server does the NAT and has 2 NIC's.

I would only need to count traffic on the external interface.  So if a
machine behind the firewall surfs the web, it would count all HTTP
packets in and out of the external interface, regardless of where the
source is.

Here's what I've got so far.

# uname -a
FreeBSD domain.com 5.4-STABLE FreeBSD 5.4-STABLE #3: Sat Oct  1 18:14:26
MDT 2005 root at domain.com:/usr/obj/usr/src/sys/CUSTOM i386

RuleSet:  I do plan on setting up ALTQ and source state limitations, but
it's not important now.  So if the code that's in there is messing
things up, it can be removed.

=================================================================================
# Interfaces
int_if = "dc0"  <--- 192.168.0.1 / 24
ext_if = "de0"  <--- Dynamic IP from ISP.

# Services 
tcp_ftp = "{ 20, 21 }"
tcp_ssh = "22"
tcp_smtp = "25"
tcp_dns = "53"
tcp_http = "80"
tcp_pop3 = "110"
tcp_https = "443"
tcp_vnc = "{ 5801, 5802 }"
tcp_bittorrent = "6800:7000"
udp_dns = "53"

# Internal Subnet
internal_net="192.168.0.0/23"  <-- Might want another /24 someday.

# Non Routable IP Addresses
table <priv_nets> persist file "/etc/bogons.txt"
# NOTE: bogons.txt contains a list of non-routable IP's from ARIN.
# One network/mask per line.  Mask is in slash notation.

# Options: tune the behavior of pf.  <--- I pulled these off the net.
set timeout { interval 10, frag 30 }
set timeout { tcp.first 120, tcp.opening 30, tcp.established 86400 }
set timeout { tcp.closing 900, tcp.finwait 45, tcp.closed 90 }
set timeout { udp.first 60, udp.single 30, udp.multiple 60 }
set timeout { icmp.first 20, icmp.error 10 }
set timeout { other.first 60, other.single 30, other.multiple 60 }
set timeout { adaptive.start 6000, adaptive.end 12000 }
set limit { states 12000, frags 5000 }

# Logs stats on the external interface.
set loginterface de0

# More default values.
set optimization normal
set block-policy drop
set require-order yes
set fingerprints "/etc/pf.os"
set debug urgent

# Normalization: reassemble fragments.
scrub on de0 all reassemble tcp
scrub in on de0 all fragment reassemble

# Queueing: rule-based bandwidth control.
#altq on $ext_if bandwidth 2Mb cbq queue { dflt, developers, marketing }
#queue dflt bandwidth 5% cbq(default)
#queue developers bandwidth 80%
#queue marketing  bandwidth 15%

# Network Address Translation with FTP Fix
nat on $ext_if from $internal_net to any -> ($ext_if)
rdr on $int_if proto tcp from any to any port ftp -> 127.0.0.1 port 8021

# Filtering: Last Matching Rule Wins.  So the defaults are at the top.
block in log all label "Blocked"

# Allow all internal traffic out.
pass in  on $int_if from $internal_net to any keep state label "Int In"
pass out on $int_if from any to $internal_net keep state label "Int Out"

# Allow all traffic from the box out and keep the state.
pass out proto { tcp, udp, icmp } from any to any keep state label
"Catch All Out $proto"

# Don't firewall the loopback interface.
pass quick on lo0

# Block non-routable IP's on the external interface
block in log quick on $ext_if from <priv_nets> to any label "Bogons In"
block out log quick on $ext_if from any to <priv_nets> label "Bogons
Out"

# NOTE: These are the 2 setup's I've tried.  
# Tried FTP without in/out specification.
# Tried SSH with in/out specification.
# I also tried without keeping state, but that didn't work either.

# FTP
pass proto tcp to any port $tcp_ftp flags S/SA keep state (max 32000,
source-track rule, max-src-nodes 75, max-src-states 6) label "FTP In"
pass proto tcp from any port $tcp_ftp keep state label "FTP Out"

# SSH
pass in proto tcp to any port $tcp_ssh modulate state (max 32000,
source-track rule, max-src-nodes 75, max-src-states 6) label "SSH In"
pass out proto tcp from any to any port $tcp_ssh keep state label "SSH
Out"

<...  Continues on with the other protocols defined at the top ...>

# Make the box pingable.
icmp_types = "echoreq"
pass in quick inet proto icmp all icmp-type $icmp_types keep state label
"ICMP In"
pass out quick inet proto icmp all icmp-type $icmp_types keep state
label "ICMP Out"

===========================================================================


Any help is surely appreciated.  I remember IPF having a "count"
command, which didn't actually filter traffic like the "pass/block"
commands, but just counted traffic.  I guess PF doesn't have that.  I'd
prefer not to use an external program to count traffic.  I'm hoping
there's someway PF can automagically do this.

Thanks in Advance.
Tyler

Here's a snippet of "pfctl -sl"

# So, no FTP traffic, which is correct.
FTP In 10139 0 0
FTP In 6687 0 0
FTP Out 6687 0 0
FTP Out 6687 0 0

# I've been SSH'd from my internal machine to the firewall all night.
Considering I didn't tell PF what interface to pass out on, I thought
SSH Out would increment.
SSH In 6687 1859 241590
SSH Out 1030 0 0

# Being on an ISP's dynamic IP block, I don't send mail via the
firewall, but I'd think there would be more than 28 packets out.
SMTP In 6687 1078 301740
SMTP Out 1018 28 6407

# Didnt do any zone transfers tonight.
DNS-TCP In 6651 0 0
DNS-TCP Out 1017 0 0

# This is definitely not right.
HTTP In 6651 23611 19562724
HTTP Out 6651 0 0


Thanks Again,
Tyler