Intel 82574L interface wedging on em 7.1.9/7.2.3 when MSIX enabled

[Jason Wolfe]

On Fri, Oct 7, 2011 at 12:24 PM, Arnaud Lacombe <lacombar at gmail.com> wrote:

> Hi,
>
> On Fri, Oct 7, 2011 at 2:57 PM, Jason Wolfe <nitroboost at gmail.com> wrote:
> > Jack,
> >
> > Entirely possible there are multiple moving pieces here, the only bit I
> know
> > for certain is it's related to the different operation when running with
> MSI
> > vs MSI-X. Here is also my loader.conf for reference. I'm currently
> running
> > the modular congestion control stuff with cubic in use, but these issues
> > predate those changes also. Just to give you a scope of it though, it was
> > somewhat 'rare' for them to wedge. Out of a pool of ~2000 servers running
> > with the 82574L doing ~800Mb/s average, there were ~220 reports in a
> week.
> > So with some fuzzy math to put it in the same terms you were talking in,
> a
> > server in particular would hang about once every 9 weeks.
> >
> Just a two questions out of my mind:
>
> Are the failing server evenly distributed, or always the same are failing ?
>
> Did you collect the uptime and the kernel msgbuf of the server when
> the issue triggered ?
>
> Thanks,
>  - Arnaud
>

Arnaud,

The failures were pretty random, though there were a handful of servers that
did fail a couple times.  It didn't seem attributable to a certain batch or
physical location.

The uptime was not collected, but most were in the ballpark of 30-90 days.
 I was tailing /var/log/messages, but didn't save kern.msgbuf no.  I've
added both of these to the collections and pulled a couple that did fail
more than once and will be re enabling MSI-X on them later today.

Jason