em driver input errors

[alexpalias-bsdnet at yahoo.com]

Hello.

--- On Wed, 8/19/09, Дмитрий Замураев <gigabyte.tmn at gmail.com> wrote:

> From: Дмитрий Замураев <gigabyte.tmn at gmail.com>
> Subject: Re: em driver input errors
> To: alexpalias-bsdnet at yahoo.com
> Cc: freebsd-net at freebsd.org
> Date: Wednesday, August 19, 2009, 7:27 PM
> Hello Alex.
> 
> What sheduler are you using? ULE or 4BSD
> Have you NIC IRQ sharing with other hardware?
> What HZ value? 1000?

SCHED_ULE, HZ=1000:

host# sysctl kern.sched.name kern.hz
kern.sched.name: ULE
kern.hz: 1000
host#

> > Thanks for the suggestion.
> > From a "clean" box:
> > dev.em.0.rx_int_delay: 0
> > dev.em.0.tx_int_delay: 66
> > dev.em.0.rx_abs_int_delay: 66
> > dev.em.0.tx_abs_int_delay: 66
> > I reset all the values (errors still appearing), then
> tried your suggestion (rx_int_delay=600,
> rx_abs_int_delay=1000).  This has reduced the number of
> >interrupts for em0 (from about 7200/sec to around
> 6500/sec).  After some time, I started getting errors
> again.
> mmm, try the maximum value 67108, what hapens...

I will try this today when there's enough traffic to see errors.

> > But that has made me try this also:
> > dev.em.0.tx_int_delay=600
> > dev.em.0.tx_abs_int_delay=1000
> I think it's a bad idea, but don't know because:
> > Meaning using your suggested values for tx too. 
> Now em0 is seeing about 1800 interrupts/second, which is way
> better, but after some time I saw errors >again...
> 
> > From the output of "netstat -nI em0 -w 5":
> maybe mistake, did you meen "netstat -w5 em0" ?

Nope, exactly as in my mail, "netstat -nI em0 -w 5".  It does take 5 seconds to produce meaningful output.

> I have PPPoE concenrator based on S3000AHV motherboard with
> Core2Quad 6600 and four (to load all cores in CPU) Intel
> PCI-E x1 and PCI-E x4 NIC's
> My load:

Pretty impressive figures.  And "netstat -ni" shows 0 errors on all cards?

> And i have't any problems. I think i select the good
> hardware.
> 
> > Interrupts total (as reported by systat):  around
> 13500/second.  I would estimate the old IRQ load at
> around 30000-35000/second, which doesn't seem too >much
> to me, for a dual xeon machine.

> I think it depends by motherbord, what full hardware
> specification are you using? with chips names

The machine is a Dell PowerEdge 2850.  According to its specs, the chipset is Intel E7520.  Two 64-bit Xeon processors at 3.20GHz, 4 GB RAM.
 
> > Speaking of which, I did compile the kernel with
> "options DEVICE_POLLING", but enabling polling only made the
> errors appear more often, and in greater >numbers.
> I don't use polling on FBSD 7.x, it's usable on FBSD older
> versions

I tried as many possibilities as I could.
 
> > - 1 x dual-port gigabit interface, PCI-X
> Maybe I have this card. And it works unstable, i don't
> remember what happens, but i seen by tcpdump "truncated IP,
> missing XX bytes"

Currently most errors are on the motherboard-embedded em0 interface.  Second is embedded em1.  Last are em2 and em3 which are on the dual-port card (em2 under 170k errors as opposed to 2.6M for em0, and em3 0 errors)
> Good luck. 

Thanks!