dummynet, em driver, device polling issues :-((

[Ferdinand Goldmann]

Kevin Day wrote:

> In one case, we had a system acting as a router. It was a Dell PowerEdge 
> 2650, with two dual "server" adapters. each were on separate PCI busses. 
> 3 were "lan" links, and one was a "wan" link. The lan links were 
> receiving about 300mbps each, all going out the "wan" link at near 
> 900mbps at peak. We were never able to get above 944mbps, but I never 
> cared enough to figure out where the bottleneck was there.

944mbps is a very good value, anyway. What we see in our setup are throuput 
rates around 300mbps or below. When testing with tcpspray, throughput hardly 
exceeded 13MB/s.

Are you running vlans on your interface? Our em0-card connects several sites 
together, which are all sitting on separate vlan interfaces for which the em0 
acts as parent interface.

> This was with PCI-X, and a pretty stripped config on the server side.

Maybe this makes a difference, too. We only have a quite old xSeries 330 with 
PCI and a 1.2GHz CPU.

> 
> Nothing fancy on polling, i think we set HZ to 10000

Ten-thousand? Or is this a typo, and did you mean thousand?

This is weird. :-( Please, is there any good documentation on tuning device 
polling? The man page does not give any useful pointers about values to use for 
Gbit cards. I have already read things about people using 2000, 4000HZ ... Gaaah!

I tried with 1000 and 2000 so far, without good results. It seems like 
everybody makes wild assumptions on what values to use for polling.

>, turned on 
> idle_poll, and set user_frac to 10 because we had some cpu hungry tasks 
> that were not a high priority.

I think I red somewhere about problems with idle_poll. How high is your 
burst_max value? Are you seeing a lot of ierrs?

*sigh* :-( confusing.

-- 
 >> Ferdinand Goldmann                                   ////  |          |
 >> EMail:  Ferdinand.Goldmann at zid.uni-linz.ac.at       |--00  |    UNIX  |
 >> Tel. : +43/732/2468/9398 Fax. : +43/732/2468/9397   C   ^  |          |
 >> EMail:  Ferdinand.Goldmann at zid.uni-linz.ac.at        \ ~/  ~~~|~~~~~~~~
 >> PGP D4CF 8AA4 4B2A 7B88 65CA  5EDC 0A9B FA9A 13EA B993| |-----3