bge dropping packets issue

[Alexander Sack]

On Wed, Apr 16, 2008 at 2:56 PM, Jung-uk Kim <jkim at freebsd.org> wrote:
> [CC trimmed]
>
>
>  On Wednesday 16 April 2008 02:20 pm, Alexander Sack wrote:
>  > Dieter: Thanks, at 20Mbps!  That's pretty aweful.
>  >
>  > JK:  Thanks again.  Wow, I searched the list and didn't see much
>  > discussion with respect to bge and packet loss!  I will try the
>  > rest of that patch including pushing the TCP receive buffer up
>  > (though I don't think that's going to help in this case).  The
>  > above is based on just looking at code....
>  >
>  > I guess some follow-up questions would be:
>  >
>  > 1)  Why isn't BGE_SSLOTS tunable (to a point)?  Why can't that be
>  > added the driver?  I noticed that CURRENT has added a lot more
>  > SYSCTL information.  Moreover it seems the Linux driver can set it
>  > up to 1024.
>
>  IIRC, Linux tg3 uses one ring for both standard and jumbo.

I'm talking about the number of slots within the ring not the number
of RX queues.

I believe the bnx4 driver (thought the tg stuff was deprecated??) uses
4 rings (one for each port perhaps) and reads hardware register at ISR
time to flip between them.

The BGE_SLLOTS define I think could be a tunable which would allow
someone to tune the driver to use more or less RX slots.  I still feel
that is a good idea instead of forcing someone to recompile! :(!

>  > 2)  bge_tick() uses the same global mutex for its callout as the
>  > rest of the driver.  Moreover, it really doesn't have to hold it
>  > while updating statistics, they are reads of volatile registers
>  > anyway (blocking the ISR doesn't prevent the firmware from updating
>  > its stat struct).  Would there be any interest in using a separate
>  > mutex for the callout itself and then just hold the lock for the
>  > other small calls (bge_asf_driver_up(), bge_watchdog())?  I'm
>  > experimenting with right now just dropping the BGE mutex around the
>  > bge_stats_update() calls to give more time to bge_rxeof() to drain
>  > rx_bd's.  I admit that bge_tick doesn't do a whole lot but it seems
>  > this driver is very sensitive to resource starvation and I'm trying
>  > to get the BGE driver to drain as much rx_bd's as possible to avoid
>  > drops due to the firmware having no place to put them!

I'd like to experiment some more with this to see if I can get 100MBps
100% non-dropping (may not be possible with small changes but it'll be
a character building experience none the less).

>  > 3)  How does interrupt non-DEVICE_POLLING perform?
>
>  If you just use default values, it won't perform very well.  There are
>  some patches around the net to automatically adjust these numbers,
>  e.g.,
>
>  http://docs.freebsd.org/cgi/mid.cgi?20071117194615.L67319
>  http://mail-index.netbsd.org/tech-kern/2004/03/16/0000.html

Thanks will take a look!

This really sucks... :D!

-aps