Intel 82550 Pro/100 Ethernet and Microcode

[YongHyeon PYUN]

On Sun, Mar 18, 2012 at 11:40:46PM +0100, Andreas Longwitz wrote:
> YongHyeon PYUN wrote:
> > 
> > The microcode is normally used to reduce high number of interrupts
> > under heavy network load by bundling multiple RX frames.
> 
> ok, I understand. But I have DEVICE_POLLING and HZ=1000 in the kernel
> source, so interrupts do not occur.
> 

Good data point. I'll try that on my box.

> > However
> > your reason to use microcode for i82550C looks weird since the
> > microcode used for i82550C does not have a fix for TCO bug.
> > The microcode for i82550(fxp2 in your system) indeed has fix for
> > TCO bug and includes additional feature for bundling.
> 
> According to the comments in rcvbundl.h it is just directly opposed.
> 
> > If you're
> > suffering from TCO bug of i82550, NFS over UDP issue should happen
> > only on i82550(fxp2). Can you check whether the NFS issue happens
> > on i82550C (fxp0 and fxp1) without loading the microcode?
> 
> The NFS issue happens on fxp0 (i82550C) without loading of microcode:
> Mar 16 16:12:52 <kern.info> dssresv1 kernel: nfs server
>                 dsscam:/prod/mobotix: not responding
> Mar 16 16:13:24 <kern.info> dssresv1 kernel: nfs server
>                 dsscam:/prod/mobotix: not responding
> Mar 16 16:13:56 <kern.info> dssresv1 kernel: nfs server
>                 dsscam:/prod/mobotix: not responding
> Mar 16 16:14:28 <kern.info> dssresv1 kernel: nfs server
>                 dsscam:/prod/mobotix: not responding
> Mar 16 16:14:59 <kern.info> dssresv1 kernel: nfs server
>                 dsscam:/prod/mobotix: is alive again
> 

I didn't ever try NFS on i82550C. If it can't handle fragmented IP
datagrams, it would also have failed netperf UDP stream test since
all UDP datagrams are fragmented.

> The last message appears immediately after loading the microcode with
>             ifconfig fxp0 link0.
> 
> > I still can't explain why your i82550C with the loaded microcode
> > does not generates SCB timeouts because mine always shows the error
> > right after loading the microcode.
> 
> In FreeBSD 6 and 8 I need my bzero-patch to get fxp working, In FreeBSD
> 4 - on the same hardware - this was not necessary.
> 
> > Are you actively using fxp0 or fxp1 after loading the microcode?
> 
> Yes I have 6 server with fxp0/fxp1 i82550C and 2 server with fxp0/fxp1
> i82550 on motherboard in production, additionally some fxp2 i82550
> external cards. All run with microcode loaded and polling.
> 
> > If yes, could you check whether
> > the CPU Saver feature of the microcode really works on i82550C?
> > You may be able to use netperf UDP stream test to verify that.
> 
> I did some tests with netperf 2.5.0, At first I had to patch some format
> strings in netperf from %d to %lld for uint64_t variables to get
> readable output from netperf on my i386 systems.
> 

Yeah, that is one of bug of netperf.

> The test command
>       netperf -H host_with_fxp -t UDP_STREAM
> gives nearly always the same output
> 
> Socket  Message  Elapsed      Messages
> Size    Size     Time         Okay Errors   Throughput
> bytes   bytes    secs            #      #   10^6bits/sec
> 
>   9216    9216   10.00       13069 1515880     96.34
>  41600           10.00       13069             96.34
> 
> And this output did not considerably change for the test cases fxp-type
> i82550C or i82550, microcode loaded or not and polling yes or no.
> 
> But I can answer your question concerning the CPU Saver funktion on
> i82550C and its a little suprise (for me). In all my tests without
> polling I checked the irq's on host_with_fxp using "vmstat -i". The
> result is that the CPU Saver feature of the microcode does not work on
> i82250C and it works on i82250. On i82250C the value of
> dev.fxp.0.bundle_max is irrelevant, the i82250C always needs ca. 91000
> irq's for the netperf test. The same number of irq's needs the i82250
> with bundle_max=1, but when I set bundle_max=6 (the default), then the
> number of irq's for the same test goes down to 91000/7 = 13000.

Thanks for the detailed information.  This indicates the microcode
for i82550C does not have bundling feature.  By chance, did you
ever update your controller firmware to newest one?  I don't
remember whether I also updated controller firmware for i82550C but
I used to update several i82559 controllers for PXE.

> 
> 
> Regards
> 
> Andreas Longwitz
>