Too much interrupts on ixgbe

[Sergey Saley]

Jack Vogel wrote:
> 
> On Tue, Oct 25, 2011 at 12:22 AM, Sergey Saley <sergeysaley@>wrote:
> 
>>
>> Ryan Stone-2 wrote:
>> >
>> > On Mon, Oct 24, 2011 at 3:51 PM, Sergey Saley <sergeysaley@>
>> wrote:
>> >> MPD5, netgraph, pppoe.Types of traffic - any (customer traffic).
>> >> Bying this card I counted on a 3-4G traffic at 3-4K pppoe sessions.
>> >> It turned to 600-700Mbit/s, about 50K pps at 700-800 pppoe sessions.
>> >
>> > PPPoE is your problem.  The Intel cards can't load-balance PPPoE
>> > traffic, so everything goes to one queue.  It may be possible to write
>> > a netgraph module to load-balance the traffic across your CPUs.
>> >
>>
>> OK, thank You for explanation.
>> And what about the large number of interrupts?
>> As for me, it's too much...
>> irq256: ix0:que 0              240536944       6132
>> irq257: ix0:que 1               89090444       2271
>> irq258: ix0:que 2               93222085       2376
>> irq259: ix0:que 3               89435179       2280
>> irq260: ix0:link                       1          0
>> irq261: ix1:que 0              269468769       6870
>> irq262: ix1:que 1                 110974          2
>> irq263: ix1:que 2                 434214         11
>> irq264: ix1:que 3                 112281          2
>> irq265: ix1:link                       1          0
>>
>>
> How do you decide its 'too much' ?  It may be that with your traffic you
> end
> up
> not being able to use offloads, just thinking. Its not like the hardware
> just "makes
> it up", it interrupts on the last descriptor of a packet which has the RS
> bit set.
> With TSO you will get larger chunks of data and thus less interrupts but
> your
> traffic probably doesn't qualify for it.
> 

It's easy. I have several servers with a similar task and load.
About 30K pps, about 500-600M traffic, about 600-700 pppoe connections. 
One difference - em
Here is a typical vmstat -i

point06# vmstat -i
interrupt                          total       rate
irq17: atapci0                   6173367          0
cpu0: timer                   3904389748        465
irq256: em0                   3754877950        447
irq257: em1                   2962728160        352
cpu2: timer                   3904389720        465
cpu1: timer                   3904389720        465
cpu3: timer                   3904389721        465
Total                        22341338386       2661

point05# vmstat -i
interrupt                          total       rate
irq14: ata0                           35          0
irq19: atapci1                   8323568          0
cpu0: timer                   3905440143        465
irq256: em0                   3870403571        461
irq257: em1                   1541695487        183
cpu1: timer                   3905439895        465
cpu3: timer                   3905439895        465
cpu2: timer                   3905439895        465
Total                        21042182489       2506

point04# vmstat -i
interrupt                          total       rate
irq19: atapci0                   6047874          0
cpu0: timer                   3901683760        464
irq256: em0                    823774953         98
irq257: em1                   1340659093        159
cpu1: timer                   3901683730        464
cpu2: timer                   3901683730        464
cpu3: timer                   3901683730        464
Total                        17777216870       2117


--
View this message in context: http://freebsd.1045724.n5.nabble.com/Too-much-interrupts-on-ixgbe-tp4931883p4935427.html
Sent from the freebsd-net mailing list archive at Nabble.com.