Sporadic TCP/RST sent to client

[sthaug at nethelp.no]

> Imagine this set up :
> 
> freebsd host port0 <-> switch 1 <-> linux host port0
> freebsd host port1 <-> switch 2 <-> linux host port1
> 
> On the linux box, port 0&1 are enslaved in a bond with a RR algorithm (Round Robin)
> On the freebsd box, port 0&1 are enslaved in a lagg.
> 
> switchs 1&2 are configured for doing MLAG.
> 
> The Linux box disapatchs packets on both NICs (since the RR algo dictates that) packets are dispatched in order.
> Packets outgoing on port0 gets handled by switch1 and hits the freebsd box on port 0
> Packets outgoing on port1 gets handled by switch2 and hits the freebsd box on port 1
> 
> As I stated earlier, from the tcpdump traces I've done on the freebsd box (both on the lagg interface and the actual ports) packets do arrive ordered but on different NICs and it works great until the elapes times start to be around microsecond.
> 
> I don't really have control over the Linux box to make them use other hash algo (but I'm stil trying)

If the Linux box is using round robin you shouldn't expect to be able
to "fix" the problem at the FreeBSD end.

On routers and switches (which is what I normally work with) the hash
algorithm used for LAG connections ensures that one "flow" always uses
the same path, thus no reordering. A typical hash algorithm uses a
5-tuple with (src ip, src port, dst ip, dst port, protocol) as input.

So the advice in this case is simple - don't use round robin! Yes, I
understand you don't control the Linux box.

Steinar Haug, Nethelp consulting, sthaug at nethelp.no