Unrecognized Inifiniband HCA

[Justin Clift]

On 2018-05-10 10:21, Grzegorz Junka wrote:
> On 08/05/2018 23:41, Justin Clift wrote:
<snip>
>> That's probably a ConnectX (series 1) Mellanox card.  Those can 
>> operate in
>> either Infiniband mode, or Ethernet mode.
>> 
>> Which mode are you wanting it to run in? :)
>> 
>> As a thought, the FreeBSD wiki page has a bit of info:
>> 
>>   https://wiki.freebsd.org/InfiniBand
>> 
>> For that card to be recognised at all, it'll need the mlx4 driver(s) 
>> to load.
>> 
>> I don't remember the exact one off hand (it's been a while), but some 
>> searching
>> online for mlx4 and FreeBSD should turn up the right bits.
> 
> Many thanks Justin. This is the first time I am hearing about an
> Infiniband card operating in Ethernet mode. These cards come with two
> CX4/SFF 8470 ports. It's not possible to connect standard Ethernet
> cables that I know of (not even SFP modules). Do you mean that they
> can operate in Ethernet mode over the CX4 cable?

Yep. :)

Back in the day when these cards were current tech, CX4 was an 
acceptable
connector for 10GbE.  The Infiniband switches from that era (that I had
access to) were mostly Infiniband only though.

But 10GbE CX4 switches did exist, and can still be found reasonably 
cheaply
on Ebay.  eg:

   * HP ProCurve 6 port CX4 10GBe switch
     https://www.ebay.com/itm/152232294328

   * HP ProCurve 48 port 1GbE switch, with 2x 10GbE CX4 ports on the back
     https://www.ebay.com/itm/281899832599

One of the good things about those HP ProCurve switches... being 
enterprise
gear they just keep working.  For Years.

 From memory, HP still releases firmware security updates for them (for
free) to this day. Unlike (say) Cisco. ;)

Note - As a data point, FreeNAS (based on FreeBSD) includes the 10/40GbE 
driver
for these cards by default.  With FreeNAS they work as 10GbE "out of the
box". :D

Hopefully that helps. :)

+ Justin