em(4) high latency w/o msix

[Arnaud Lacombe]

Hi,

On Sun, Oct 2, 2011 at 8:29 PM, Arnaud Lacombe <lacombar at gmail.com> wrote:
> Hi,
>
> On Sun, Oct 2, 2011 at 12:08 PM, Jack Vogel <jfvogel at gmail.com> wrote:
>> On what hardware?
>>
> Only the 82574 is using em(4)'s MSI-X.
>
FWIW, I'm not seeing this issue on a 82574 box I've access to:

result of the following command:

# ping -nqt 30 -f <ip>

The two box are connected on an old 10/100 switch

MSI-X enabled:
239361 packets transmitted, 239358 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 0.090/0.117/2.488/0.027 ms

MSI-X disabled:
233122 packets transmitted, 233119 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 0.096/0.120/2.589/0.027 ms

On another interface (same boxes involved, but through a 10/100/1000 switch):

MSI-X enabled:
240080 packets transmitted, 240079 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 0.090/0.116/0.175/0.008 ms

MSI-X disabled:
207310 packets transmitted, 207309 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 0.095/0.136/0.310/0.044 ms

However, there seem to be a performance drop with MSI-X disabled and
Gigabit link.

 - Arnaud

>> Jack
>>
>>
>> On Sun, Oct 2, 2011 at 6:42 AM, Hooman Fazaeli <fazaeli at sepehrs.com> wrote:
>>
>>>
>>> Latest em(4) driver from HEAD seems to have high latency
>>> when MSIX is disabled.
>>>
>>> With MSIX enabled (hw.em.enable_msix=1):
>>>
>>> # ping -c5 192.168.1.83
>>> PING 192.168.1.83 (192.168.1.83): 56 data bytes
>>> 64 bytes from 192.168.1.83: icmp_seq=0 ttl=64 time=0.055 ms
>>> 64 bytes from 192.168.1.83: icmp_seq=1 ttl=64 time=0.076 ms
>>> 64 bytes from 192.168.1.83: icmp_seq=2 ttl=64 time=0.066 ms
>>> 64 bytes from 192.168.1.83: icmp_seq=3 ttl=64 time=0.051 ms
>>> 64 bytes from 192.168.1.83: icmp_seq=4 ttl=64 time=0.063 ms
>>>
>>> --- 192.168.1.83 ping statistics ---
>>> 5 packets transmitted, 5 packets received, 0.0% packet loss
>>> round-trip min/avg/max/stddev = 0.051/0.062/0.076/0.009 ms
>>>
>>> With MSIX disabled:
>>>
>>> # ping -c5 192.168.1.83
>>> PING 192.168.1.83 (192.168.1.83): 56 data bytes
>>> 64 bytes from 192.168.1.83: icmp_seq=0 ttl=64 time=0.180 ms
>>> 64 bytes from 192.168.1.83: icmp_seq=1 ttl=64 time=0.164 ms
>>> 64 bytes from 192.168.1.83: icmp_seq=2 ttl=64 time=0.169 ms
>>> 64 bytes from 192.168.1.83: icmp_seq=3 ttl=64 time=0.172 ms
>>> 64 bytes from 192.168.1.83: icmp_seq=4 ttl=64 time=0.167 ms
>>>
>>> --- 192.168.1.83 ping statistics ---
>>> 5 packets transmitted, 5 packets received, 0.0% packet loss
>>> round-trip min/avg/max/stddev = 0.164/0.170/0.180/0.005 ms
>>>
>>> As you see, w/o MSIX, RTT increases by a factor of 3.
>>>
>>> I also tested the following drivers:
>>>    - igb(4) from HEAD: OK.
>>>    - Stock 7.3-RELEASE: OK.
>>>    - Stock 7.4-RELEASE: problem exist.
>>>
>>> Any ideas?
>>>
>>>
>>>
>>>
>>>
>> _______________________________________________
>> freebsd-net at freebsd.org mailing list
>> http://lists.freebsd.org/mailman/listinfo/freebsd-net
>> To unsubscribe, send any mail to "freebsd-net-unsubscribe at freebsd.org"
>>
>