Help with igb driver/nics, strange issue.

[joe]

On 05/06/2010 02:18 PM, Ian FREISLICH wrote:
> joe wrote:
>    
>> I have 3 boxes, each with two nics. One nic for the private network and
>> one for the public network.
>> The private network is all on the same vlan. All 6 nics are on the same
>> switch. All connections are 1000tx Full Duplex.
>>
>> I will call the servers Box A, Box B, and Box C.
>>
>> When i FTP data between Box A&  B i get abou 25MB/sec.
>> When i FTP data from Box C to Box A or B, i get about 20MB/sec.
>> When i FTP data from Box A to C i get 10MB/sec
>> When i FTP data from Box B to C i get 200KB/sec...
>>
>> Can anyone suggest why i might only be getting 200KB when transfering
>> data from Box B to C but not when transferring data from Box A to C?
>>      
> Is the hardware exactly the same on all 3 hosts?  From your enumeration
> it looks like there's something special about box C.
>
> How busy are the disks?  One of the problems with FTP, at least the
> last time I tried to use it for benchmarking was that it used tiny,
> tiny transfers to and from disk.  Strangely scp did better even
> with the crypto overhead.  Have you tried using netperf to test the
> network performance?
>
> Have you checked your cables?  I've seen all sorts of wierd problems
> caused by cables.  netstat -ni should give an idea of transmission
> problems.  If the switch is a managed switch, you can also check
> its interface counters.
>
> Ian
>
> --
> Ian Freislich
>    
Hi Ian,

Thanks for the help!
The servers are mostly different. Server A & B run on consumer grade 
motherboards/ram/nics. They both have a 12 disk raid 5 setup. Server C 
is a dual quad core xeon processor supermicro server. It's Disk setup is 
raid5 using the intel raid matrix. The switch is managed but i dont have 
the login information for it (been years since i've had to log in ;(). 
The switch is a netgear gigabit switch. I've had the cables replaced as 
well as trying two different ports on the switch, and ended up with the 
same results. I've included the netstat -ni results below.  While we 
might be at a disadvantage until i get access to the managed switch, is 
there anything else we might be able to try , to resolve the problem?

Joe

Name    Mtu Network       Address              Ipkts Ierrs Idrop    
Opkts Oerrs  Coll
igb0   1500 <Link#1>      00:30:48:9f:11:04     8617     0     0     
6108     0     0
igb0   1500 216.105.91.14 216.105.91.145        7066     -     -     
6082     -     -
igb1   1500 <Link#2>      00:30:48:9f:11:05     3126     0     0      
680     0     0
ipfw0 65536 <Link#3>                               0     0     0        
0     0     0
lo0   16384 <Link#4>                               0     0     0        
0     0     0
lo0   16384 fe80:4::1/64  fe80:4::1                0     -     -        
0     -     -
lo0   16384 ::1/128       ::1                      0     -     -        
0     -     -
lo0   16384 127.0.0.0/8   127.0.0.1                0     -     -        
0     -     -
pflog 33152 <Link#5>                               0     0     0        
0     0     0
vlan1  1500 <Link#6>      00:30:48:9f:11:05     1374     0     0      
680     0    11
vlan1  1500 192.168.1.0/2 192.168.1.2            796     -     -      
677     -     -
vlan1  1500 192.168.1.210 192.168.1.210            2     -     -        
0     -     -
vlan1  1500 192.168.1.211 192.168.1.211            0     -     -        
0     -     -
vlan1  1500 192.168.1.212 192.168.1.212            0     -     -        
0     -     -
vlan1  1500 192.168.1.213 192.168.1.213            0     -     -        
0     -     -
vlan1  1500 192.168.1.214 192.168.1.214            0     -     -        
0     -     -
vlan1  1500 192.168.1.215 192.168.1.215            0     -     -        
0     -     -
vlan1  1500 192.168.1.216 192.168.1.216            0     -     -        
0     -     -
vlan1  1500 192.168.1.217 192.168.1.217            0     -     -        
0     -     -
vlan1  1500 192.168.1.218 192.168.1.218            0     -     -        
0     -     -
vlan1  1500 192.168.1.219 192.168.1.219            0     -     -        
0     -     -