Infiniband experiment environment setting up

[Garrett Cooper]

On Tue, Nov 20, 2012 at 10:59 AM, George Neville-Neil
<gnn at neville-neil.com> wrote:
>
> On Nov 18, 2012, at 21:53 , Daichi GOTO <daichi at bsdconsulting.co.jp> wrote:
>
>> Hi,
>>
>>  Currently, we are getting ready to test InfiniBands on FreeBSD
>> using follow devices:
>>
>>       CPU:            Intel Xeon E3-1240v2
>>       M/B:            Super Micro X9SCM
>>       RAM:            Kingston KVR16E11/8 (8GBx4)
>>
>>       IB Card:        Mellanox Technologies ConnectX-2 VPI MHQH19B-XTR
>>       IB Card:        Mellanox Technologies ConnectX-3 VPI MCX353A-QCBT
>>       IB Switch:      Mellanox Technologies EXW-IS5022
>>       QSFP cable:     MC22096-130-001
>>
>>  Is there some information about above InfiniBand devices?
>
> Well it's only the cards you have to worry about.  I know the ConnectX-2 works
> but have not tested the ConnectX-3.

    We're getting access to ConnectX-3 cards at Isilon soon, but I'm
not sure what sourcebase we'll be testing against (our local
sourcebase, CURRENT, or CURRENT + new OFED). My team can best speak to
what we'll be doing here.

> Also, note that it depends on what you are testing.  That is, Infiniband has low
> latency at low payloads.  The numbers quoted by Mellanox about low latency depend
> on you sending one byte (yes, 1 byte).  At real packet sizes Infiniband is no faster than
> 10GbE.

    I heard similar.
Thanks,
-Garrett