traffic shaping on RELENG_10 ?

Thu Dec 17 19:24:38 UTC 2015

On Wed, Dec 16, 2015 at 15:20:26 -0500, Mike Tancsa wrote:
> I need to do some simple traffic shaping, but whenever I try and use
> altq, I dont seem to have any luck-- I mean zero.

It took me a while to get ALTQ working.  More below..

>  I like the management of pf via pf.conf, but miss the simplicity of
> dummynet and ipfw.  Has anyone used pf and ipfw together in RELENG10 ?
> Any tips / caveats ? Or am I better off re-writing the pf rules into
> ipfw and be done with it.

I'm using ipfw for firewall and in-kernel NAT, and ALTQ (configured via pf)
for traffic shaping.

Step 1 was getting an ethernet card that supported ALTQ.  I tried enabling
IGB_LEGACY_TX in the igb driver, but it led to panics.  I tried configuring
ng_iface to provide the single queue that ALTQ needs, but wasn't able to
figure it out.  It ended up being easier to buy a $22 ethernet card that
uses the em(4) driver.

Hopefully this will help someone in a similar situation reading the list
archives...

This mostly works for me.  I have 10Mb upstream.  One of the keys was
getting outbound DNS traffic to have a real-time priority (it's in the
"other" category below) so that web browsers didn't time out when there was
heavy upstream traffic.  (e.g. uploading gigabytes of a photo library to
my iCloud Photo Library...)

One thing that was non-obvious with the in-kernel NAT versus the userland
natd(8) was that you have to set 'sysctl net.inet.ip.fw.one_pass=0'.
Otherwise the rules after the NAT step in the ipfw configuration are
skipped.

Using the in-kernel NAT was necessary to allow tagging packets from certain
internal hosts so that they could be routed to different ALTQ rules after
the NAT step.  The tags are lost if you use the userland natd.

I use ipfw count rules to tag packets, and then make decisions about
whether to drop them later.

Although the priorities are specified, I believe the hfsc schedule does not
use them.  So pay no attention to that.  And I don't currently have any
hosts tagged in the lpweb (low priority web) category.  I'm not sure that
would be productive in any case without a change in the ALTQ setup to limit
the bandwidth for that queue or something like that.  And, even then, if I
have a large upload going on, I want it to go as fast as possible when it
isn't contending with other traffic.

Here's my pf.conf:

altq on em0 hfsc bandwidth 9600Kb queue { lpweb, mail, web, hphost, ssh, icmp, femtocell, other }
queue lpweb bandwidth 20% priority 0 hfsc (upperlimit 99%)
queue mail bandwidth 10% priority 1 hfsc (upperlimit 99%)
queue web bandwidth 20% priority 2 hfsc (upperlimit 99%)
queue hphost bandwidth 10% priority 3 hfsc (upperlimit 90% realtime 20%)
queue other bandwidth 20% priority 4 hfsc (default upperlimit 99% realtime 20%)
queue icmp bandwidth 5% priority 5 hfsc (upperlimit 15%)
queue ssh bandwidth 10% priority 6 hfsc (upperlimit 99% realtime 15%)
queue femtocell bandwidth 5% priority 7 hfsc (upperlimit 50% realtime 20%)

And here are some snippets from my firewall script.  Obviously this isn't
complete:

fwcmd="ipfw"
pfcmd="pfctl"
oif="em0"
iif="igb0"
oip="my_external_ip_address"

Here is where I tag outbound packets from certain hosts.  The tags will
follow the packets through the in-kernel NAT so that I can then put them on
the correct ALTQ on the outside interface:

# Streaming boxes
${fwcmd} add count tag 10 ip from 10.0.0.0/24{30-39} to any via ${iif}

# Mark packets coming from the Verizon Femtocell
${fwcmd} add count tag 30 all from 10.0.0.44 to any via ${iif}

In-kernel NAT:

${fwcmd} nat 123 config log if ${oif} same_ports unreg_only

# This is necessary with the in-kernel NAT to re-inject packets once they've
# gone through the NAT.
sysctl net.inet.ip.fw.one_pass=0

# ALTQ queues have to be setup via pf
${pfcmd} -f $PF_FILE

# Use count rules to tag the traffic in different queues.

# Femtocell and similar super-high priority, but lower bandwidth 
# machines.
${fwcmd} add count altq femtocell ip from any to any tagged 30-39

# Tags 10-19 are high priority machines.
${fwcmd} add count altq hphost ip from any to any tagged 10-19

# ${WORK} VPN server.  ssh connections to work run over the VPN.
${fwcmd} add count altq ssh ip from any to ${WORK_VPN_SERVER} dst-port 443

# Low priority machines.  This would primarily be needed if there
# were a machine taking up a lot of bandwidth.
${fwcmd} add count altq lpweb ip from any to any dst-port 443 tagged 20-29

# IMAP, outbound mail
${fwcmd} add count altq mail ip from ${oip} 25,143,993 to any 
${fwcmd} add count altq mail ip from ${oip} to any dst-port 25

# Normal web access.
${fwcmd} add count altq web ip from any to any dst-port 80,443

# SSH
${fwcmd} add count altq ssh ip from any to any dst-port 22

# ICMP
${fwcmd} add count altq icmp icmp from any to any

# Explicitly add DNS to the other queue.
${fwcmd} add count altq other udp from any to any dst-port 53

# Anything else, especially DNS.  This is the default.
${fwcmd} add count altq other ip from any to any via ${oif}
${fwcmd} enable altq

Ken
-- 
Kenneth Merry
ken at FreeBSD.ORG