Best practice for virtualized pf based NAT router?

[Trond Endrestøl]

Hi,

I'm in the process of configuring a virtualized pf based NAT router. 
The NAT router is supposed be a supplement to our pool of public IPv4 
addresses.

FreeBSD is stable/11, r306639. XenServer 7.0.0, with all known 
updates, is the virtualization environment.

I'm using xn0 as the external interface, and xn1 as the internal 
interface.

The xn0 interface has a /30 IPv4 address and a /64 IPv6 address.
The xn1 interface has a /20 IPv4 address (and a /64 IPv6 address for symmetry).

I followed ch. 29.3.3.1 of the Handbook.

In theory all is well, but with iftop(8) (net-mgmt/iftop) I only see a 
throughput of merely 1 Mbit/s, yes, that's one megabit per second.

Running fetch(1) and ftp(1) directly on the NAT router gives me far 
better speeds, anything from 480 Mbit/s to 720 Mbit/s.

My /etc/pf.conf file looks like this:

### 8< ###################### snip ################################ >8

# From the example in the Handbook, ch. 29.3.3.1.

# Macros:
ext_if="xn0"
int_if="xn1"
localnet = $int_if:network

# Rules:
nat on $ext_if from $localnet to any -> ($ext_if)
block all
pass from { lo0, $localnet } to any keep state

# My own stuff:

# Should I restrict any non-NAT44 traffic or let it all pass?
pass all

# Allow IPv6 everywhere.
# Maybe not reasonable for a NAT44 GW, but it's not acting as an IPv6 GW.
#pass inet6 all

# We should allow SLAAC on $int_if.
# Maybe this rule is too generous.
#pass on $int_if inet6 keep state

# These rules allows the GW to talk to outsiders via $ext_if.
# Maybe the rules are too generous.
#pass inet  from $ext_if to any keep state
#pass inet6 from $ext_if to any keep state

### 8< ###################### snip ################################ >8

Does anyone have any advice on how to achieve better throughput?

I'm not new to FreeBSD, but pf is an unknown territory. My last 
attempt at doing NAT was with IPFW and natd(8) running FreeBSD 4 or 5 
on a physical computer, some 15 years ago. Any advice will be highly 
appreciated.

-- 
+-------------------------------+------------------------------------+
| Vennlig hilsen,               | Best regards,                      |
| Trond Endrestøl,              | Trond Endrestøl,                   |
| IT-ansvarlig,                 | System administrator,              |
| Fagskolen Innlandet,          | Gjøvik Technical College, Norway,  |
| tlf. mob.   952 62 567,       | Cellular...: +47 952 62 567,       |
| sentralbord 61 14 54 00.      | Switchboard: +47 61 14 54 00.      |
+-------------------------------+------------------------------------+