Packets passed by pf don't make it out?

[Kristof Provost]

On 12 Oct 2020, at 23:48, Andreas Longwitz wrote:
> Hello,
>
> now I can confirm (on FreeBSD 10 Stable) what you see on fb2 when your
> program udp_client is running on fb1. pf creates a state for the first
> packet only, for the other packets pf failes to create a state with
> messages like
>
> pf: stack key attach failed on re0: UDP in wire: 192.168.14.10:23456
> 172.16.0.2:12345 stack: 192.168.14.10:23456
> 192.168.14.100:12345 1:0, existing: UDP in wire: 192.168.14.10:23456
> 172.16.0.1:12345 stack: 192.168.14.10:23456 192.168.14.100:12345 1:0
>
> pf gives this messages in debug mode (pfctl -x loud).
>
> I do not know if we see a bug in pf or if your program udp_client does
> something illegal, I think Kristof can tell us.
>
Your confidence is both flattering and misplaced :)

I think I can reproduce the problem on CURRENT and with VNET jails, 
which is convenient.

I see the same ‘stack key attach failed’ error message. My current 
thinking is that we’re hitting a state collision, because post-RDR our 
connection information is the same (192.168.14.10:23456 
192.168.14.100:12345). That means we can’t create a new state, and the 
packet gets dropped.

It’s a little unusual for a client to keep re-using src ports like 
that, but it’s not actually wrong.
I’m not sure how we can fix this.

Best,
Kristof