dummynet and vr(4)/egress broken in 4.11 ?

[Jeremie Le Hen]

Hi,

I've been using DUMMYNET for two years on RELENG_4.  It worked quite
well until I upgrade to 4.11 yesterday.  I first thought it was due to
some error in my rule file since it is quite complex : each outgoing
packets goes indeed through one queue for traffic scheduling and
multiple pipes for bandwidth resevation (this configuration is so
powerful that I didn't have to switch to ALTQ yet).

FYI, my packet filter is ipf(8), and I use ipfw(8) for traffic shaping
only.

Weirdly, when I try to go to establish a TCP connection to some host
on Internet, I am able to resolve its name, the SYN packet successully
reach its destination, I get the SYN/ACK but the final ACK packet of the
3WHS is blocked (dropped ? sent is orbit ?) by my FreeBSD 4.11 routern.
As far as I tested, this happens to all TCP connections concerning hosts
inside my network (which are NATed), but it works perfectly from the
FreeBSD router itself.

At first glance, this problem looked like a MTU issue, but flushing all
ipfw rules makes things work correctly.  I tried disabling rules step by
step to narrow the problem, but it persists until I remove the last
DUMMYNET pipe, whichever it is.  Thus I flushed all rules and just used
(217.12.3.11 is yahoo.fr) :

%%%
    # ipfw pipe 1 config bw 10 Kbytes/s
    # ipfw add pipe 1 tcp from any to 217.12.3.11 out xmit vr0
%%%

and the same problem happened !

I didn't changed my kernel configuration file so much since my last
kernel upgrade, I juste added gif(4), IPSEC_FILTERGIF and vr(4).
I tested using this rule on ingress and egress of both my internal (sis0)
and external interface (vr0) - inverting IPs where needed :-) - here are
the results :

           | ingress | egress  |
-----------+---------+---------+
vr0 (ext)  |   OK    |    -    |
-----------+---------+---------+
sis0 (int) |   OK    |   OK    |
-----------+---------+---------+

I think that it is now very important to tell you that while upgrading
my box to FreeBSD 4.11, I also changed my external interface from a 10
MBits ep(4) to a 100 MBits vr(4).

I cannot switch back to ep(4) for the moment since it is not an option
to have downtime, but according to the privous results, I'm pretty
convinced there is a problem with the vr(4) driver (although I don't
know how it can impact DUMMYNET).  Maybe the last commit on this
driver in RELENG_4 (sys/pci/if_vr.c, rev 1.26.2.14) is the culprit.

Best regards,
-- 
Jeremie Le Hen
jeremie at le-hen.org