Dummynet, pipes and VNET jails

[Stefan.Erl at dlr.de]

Hi all,

I have a problem with ipfw/dummynet, pipes and VNET jails using FreeBSD 12.0
release. Packets are lost in the pipe when any impairments are configured.

I set up several VNET jails and connected them via epairs, in order to 
do tests with different network and routing configurations. On some jails, I
want to run dummynet with pipes to add delay to the packets. Whenever
I configure a pipe with zero delay, everything works. As soon as I add any delay
(or loss, bw limit), the packets enter the pipe, but never exit it, and never
appear on the target Jail.


A simple test setup I've set up is the following:
(JailB is configured for IP forwarding)

---------     ---------     ---------
| JailA |-----| JailB |-----| JailC |
---------     ---------     ---------

JailA: ping JailC

JailB: 
	ipfw flush
	ifpw add 10000 pipe 1 ip from JailA to any
	ipfw config pipe 1 delay 0ms
	
	This works fine, packets arrive at JailC. "ipfw show" shows increasing
	counters for rules 10000 and 65535 (allow ip from any to any)
	
	Then, if I add some delay:
	
	ipfw config pipe 1 delay 10ms
	
	Packets are lost in JailB, don't arrive at JailC. "ipfw show" shows
	increasing counters only for rule 10000, but not for 65535.
	

IPFW and dummynet are compiled into the kernel, with kern.hz=1000 and 
IPFW_DEFAULT_TO_ACCEPT option. Dummynet pipes on the Jail ethernet interface
outside a jail seem to work fine.

I've found some threads from 2010 saying that there are some problems with
dummynet and VNET jails. Are these still existing, is something else wrong, or
am I missing something?

Regards,
Stefan