Dropped vs. missed packets in the ixgbe driver

[Barney Cordoba]

--- On Mon, 8/17/09, Manish Vachharajani <manishv at lineratesystems.com> wrote:

> From: Manish Vachharajani <manishv at lineratesystems.com>
> Subject: Dropped vs. missed packets in the ixgbe driver
> To: freebsd-net at freebsd.org
> Date: Monday, August 17, 2009, 6:24 PM
> I've been doing some performance
> testing on freebsd 7.2 and noticed
> that the ixgbe driver does not report missed packets as
> dropped when
> queried via netstat -id (the ixgbe driver in the 8.0 
> beta has a
> similar issue).  A missed packet is a packet that was
> correctly
> received by the NIC but because it was out of descriptors
> and internal
> memory, the packet had to be dropped by the NIC itself -- a
> hardware
> register counts such events.  Instead the driver only
> reports drops
> that are due to the driver itself, i.e., the NIC DMAed the
> packet to
> memory but the driver had to drop something because it was
> out of
> mbufs.  Is the miss count reported elsewhere (besides
> via a kernel
> printf from the driver)?  At the end of the email I
> give a stats dump
> from the driver and from netstat, note the number of
> packets dropped
> is 0 as reported by netstat but the Missed field in the
> dmesg output
> shows many missed packets.
> 
> >From my perspective, it is disconcerting to see
> performance
> degradation on the link, along with TCP ack retransmits,
> packet
> reordering, etc. (on a point-to-point link with no switch
> in between)
> but then see no drops reported by netstat because the
> driver didn't
> drop the packet, the NIC did.  The fix should be
> straight-forward and
> I'll gladly make a patch assuming that it is indeed a bug
> and not a
> conscious design choice.
> 
> Here is the relevant netstat output
> 
> Name    Mtu Network   
>    Address         
>     Ipkts Ierrs    Opkts
> Oerrs  Coll Drop
> ix0    1500 <Link#1>     
> 00:30:48:94:60:ec        0 
>    0        1
>  0     0    0
> ix0    1500 192.168.105.0 192.168.105.2 
>           0 
>    -        0
>  -     -    -
> ix1    1500 <Link#2>     
> 00:30:48:94:60:ed      11M 
>    0     6.1M
>  0     0    0
> ix1    1500
> 192.168.5.0   192.168.5.2     
>       10M     - 
>    6.1M
>  -     -    -
> 
> And here is the dmesg output after doing a sysctl
> dev.ix.1.stats=1
> 
> ix1: Std Mbuf Failed = 0
> ix1: Missed Packets = 413872
> ix1: Receive length errors = 0
> ix1: Crc errors = 0
> ix1: Driver dropped packets = 0
> ix1: watchdog timeouts = 0
> ix1: XON Rcvd = 616428212235
> ix1: XON Xmtd = 0
> ix1: XOFF Rcvd = 616428212235
> ix1: XOFF Xmtd = 0
> ix1: Total Packets Rcvd = 12424533
> ix1: Good Packets Rcvd = 12010661
> ix1: Good Packets Xmtd = 6419128
> ix1: TSO Transmissions = 0
> 
> Manish

the debug sysctl show more interesting info. Don't get too excited
about doing performance testing with that driver. Its not designed
to be any higher in performance than any of the other intel drivers.

Barney