Serious bug in most (?) ethernet drivers (bge, bce, ixgb etc.).

[Bruce Evans]

On Thu, 5 Apr 2007, Vladimir Ivanov wrote:

> We have reported serious bug with em driver 
> (http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/87418) one year and half 
> ago.
> It's very funny but most freebsd ethernet drivers cloned this bug I seem.
> You can see same bug in bce, bge, ixgb and so on.

I can only see it in bce and ixgb.  bge is much simpler and better --
bge_rxeof() doesn't depend on any state after the unlock/re-lock except
the rx indexes, and these are both reset to 0 by reinitialization.

However, reinitialization often panics bge_rxeof() anyway.   The only
reasons for the panics that I can think of is that nothing is declared
volatile but the producer index is extremely volatile, so the following
races are possible:
- compiler caching the indexes.  bce implements this as foot-shooting.
   I think aliasing problems prevent the compiler doing it, so declaring
   things as volatile would make no difference.
- a race with the hardware in initialzation might result in the producer
   index being nonzero and for old data despite it having been reset to 0
   and no new data arriving.  Stopping the hardware for initialization
   should prevent such races.

Bruce