tuning routing using cxgbe and T580-CR cards?

[Bjoern A. Zeeb]

On 12 Jul 2014, at 12:17 , Olivier Cochard-Labbé <olivier at cochard.me> wrote:

> On Fri, Jul 11, 2014 at 8:03 PM, Bjoern A. Zeeb <
> bzeeb-lists at lists.zabbadoz.net> wrote:
> 
>> If you are primarily forwarding packets (you say "routing" multiple times)
>> the first thing you should do is turn off LRO and TSO on all ports.
>> 
> 
> Hi Bjoern,
> 
> I was not aware of disabling LRO+TSO for forwarding packet.
> If I read correctly the wikipedia page of LRO[1]: Disabling LRO is not a
> concern of performance but only of not breaking the end-to-end principle,
> right ?
> But regarding TSO[2]: It should improve performance only between the TCP
> and IP layer. But paquet forwarded didn't have to cross TCP<->IP layer,
> then disabling TSO should not impact performance, right ?

For forwarding it means that you are re-assembling a packet on receive, buffering multiple, etc, then hand them up the stack, only to find that you are sending it out again, and thus you break them into multiple packets again.   In other words:  you do a lot more work and add latency than you need/want.

I seem to remember that we added the knob to automatically disable our soft-LRO when forwarding is turned on (but I haven’t checked if I really did).  If we did, at least for soft-LRO you won’t notice a difference indeed.


> - Multi-flows (different UDP ports) of small packet (60B) at about 10Mpps
> …

> No difference proven at 95.0% confidence
> 
> => There is not difference: Then I can disable LRO for respecting the
> end-to-end principle. But why disabling TSO ?

Try TCP flows.

— 
Bjoern A. Zeeb             "Come on. Learn, goddamn it.", WarGames, 1983