ipfw + bridge + epair + tags for vnet jails after upgrade to 13.1

Hi, I am trying to wrap my head around ifbridge and ipfw.

I upgraded a host from 13.0 to 13.1

Up until now, I used ipfw, bridge and epairs on 13.0 and felt like 
I knew what I am doing.  Now I can not get the system to behave 
like it should, so either my mental model is wrong or something 
changed.  Perhaps someone could help me to understand what I am 
doing wrong.


Environment
-----------

I can't have a physical interface as member of the jailbridge, 
because this leaks virtual mac addresses of epair interfaces to 
the outside world where my hoster looks unkindly on mac-addresses 
not belonging to the nic of my server.  So I have vnet jails 
behind a common ifbridge.  All jails have their default routes 
point to the bridge-interface of the host.  The host works as a 
router.


Tags stopped working across vnet and bridge
-------------------------------------------

On a long running host that is still currently running 13.0 I have 
this line in a vnet jail with an epair interface acme_j:

allow tag 128 tcp from me to any 80,443 via acme_j setup uid root 
keep-state

On the host I see the tags:

# ipfw -a list 570

00570 112 11276 count tagged 128

On the updated 13.1 machine the host does not see the tags, or I 
can't get the host to count them.


General understanding and testing
---------------------------------

Man ifbridge pointed me to pfil.  To learn and understand how 
these tools interact I put the following rules on the 13.1 host.

count log via em0
count log via bridge0
count log via epair0a

with epair0a being a member of the bridge.  If I fetch a file in 
the vnet jail containing epair0b the counters of em0 and bridge0 
increment, but the counter of epair0a does not increment.  Tcpdump 
-i epair0a does show the traffic though.

A similar count rule I left in two days ago has counted to 43 
packets 3096 since then.  How can that be?


My sysctls are
--------------

net.link.bridge.ipfw: 0
net.link.bridge.allow_llz_overlap: 0
net.link.bridge.inherit_mac: 0
net.link.bridge.log_stp: 0
net.link.bridge.pfil_local_phys: 0
net.link.bridge.pfil_member: 1
net.link.bridge.ipfw_arp: 0
net.link.bridge.pfil_bridge: 1
net.link.bridge.pfil_onlyip: 1

net.inet.ip.fw.dyn_keep_states: 1                   
net.inet.ip.fw.dyn_keepalive: 1
net.inet.ip.fw.dyn_short_lifetime: 5
net.inet.ip.fw.dyn_udp_lifetime: 10
net.inet.ip.fw.dyn_rst_lifetime: 1
net.inet.ip.fw.dyn_fin_lifetime: 1
net.inet.ip.fw.dyn_syn_lifetime: 20
net.inet.ip.fw.dyn_ack_lifetime: 300
net.inet.ip.fw.dyn_parent_max: 4096
net.inet.ip.fw.dyn_max: 16384
net.inet.ip.fw.dyn_buckets: 8192
net.inet.ip.fw.curr_max_length: 1
net.inet.ip.fw.curr_dyn_buckets: 256
net.inet.ip.fw.dyn_parent_count: 0
net.inet.ip.fw.dyn_count: 1
net.inet.ip.fw.enable: 1
net.inet.ip.fw.static_count: 51
net.inet.ip.fw.default_to_accept: 0
net.inet.ip.fw.tables_sets: 0
net.inet.ip.fw.tables_max: 128
net.inet.ip.fw.default_rule: 65535
net.inet.ip.fw.verbose_limit: 0
net.inet.ip.fw.verbose: 1
net.inet.ip.fw.autoinc_step: 100
net.inet.ip.fw.one_pass: 1

How do I control what traffic on which interfaces ipfw _should_ 
see and how do I ask it what it _actually_ sees?


net.link.bridge.pfil_onlyip
---------------------------

Does this setting mean that arp gets filtered?
When does this happen.  The docstring says when pfil is enabled. 
Does this refer to a sysctl being set, and if so which one, or 
ipfw being enabled?  I assume that ipfw uses pfil.

Do the pfil settings of net.link.bridge control how it connects to 
ipfw or am I barking up the wrong tree?


Sorry for the wild questions.  I am just very confused and 
propably just missing something very basic.

Thank you in advance.

Markus

-- 
Markus Graf

Tel.: +49 172 840 26 08
Email: markus.graf@markusgraf.net
Web: markusgraf.net, headhuntertraining.net