Mellanox Case # 00320058 : RE: Slow speed with Mellanox IS5023 Switch and Connect-X3 cards was updated [ref:_00D507heg._50050q9Npg:ref ]

Mellanox Support Admin supportadmin at mellanox.com
Mon Dec 5 20:29:40 UTC 2016


Dear Vinícius Ferrão, 

Your Mellanox support case # 00320058/   has been updated By , with the following comment: 
/-------------------------------------------------------------
Hi Benny,

I am Anderson Dias Silva (LATAM SE). Please, see below an important consideration of our customer.

###########################################

After some testing we discovered that was a mismatch between the client and the server on operating modes of IPoIB. One side was using Connected Mode (the FreeBSD one) and the Linux client was on Datagram mode).

So I’m considering running FreeBSD in UD (Datagram) mode to see if I finally can get rid of the problems. But, how can I change between CM and UD in FreeBSD? We don't have a /sys folder on FreeBSD so I cant just set from 0 to 1 and change modes.

###########################################

So, how can we change between CM and UD in FreeBSD?

Ragards!
-------------------------------------------------------------/

Thank You, 

 

Click on the link to access the case: 
https://mymellanox.force.com/support/5005000000q9Npg

Don’t forget to check out our Online Academy, register now to learn about the products and technology, http://academy.mellanox.com/login/index.php 

/------ Your Case Comments History ------ 
-------------------------------------------------
  Created by: Anderson Dias Silva 2016-12-05 20:29:35
Hi Benny,

I am Anderson Dias Silva (LATAM SE). Please, see below an important consideration of our customer.

###########################################

After some testing we discovered that was a mismatch between the client and the server on operating modes of IPoIB. One side was using Connected Mode (the FreeBSD one) and the Linux client was on Datagram mode).

So I’m considering running FreeBSD in UD (Datagram) mode to see if I finally can get rid of the problems. But, how can I change between CM and UD in FreeBSD? We don't have a /sys folder on FreeBSD so I cant just set from 0 to 1 and change modes.

###########################################

So, how can we change between CM and UD in FreeBSD?

Ragards!

-------------------------------------------------
  Created by: Benny Filin 2016-12-05 01:03:55
Hello Vinícius Ferrão,

Thank you for contacting Mellanox Technical Support.

1) DId you try changing switch ports when testing performance between two FreeBSD machines through switch?
2) What happens when you try a back-to-back connection between two FreeBSD machines? Do you get good results?
3) Did you test the performance between FreeBSD and a Linux machine through switch?

Please, follow the provided procedure in order to query your unmanaged switch: 
 
1) Download and install MFT (Mellanox Firmware Tool):
http://www.mellanox.com/content/pages.php?pg=management_tools&menu_section=34
Refer to the User Manual for the installation instructions:
http://www.mellanox.com/pdf/MFT/MFT_user_manual.pdf
 
2) From one of the nodes in the fabric, invoke: ibswitches.
Match the relevant switch GUID from the Mellanox switch label to the LID.
 
3) From one of the nodes in the fabric , invoke: mst start -> mst ib add -> mst status
 
4) Identify the relevant unmanaged switch mst representation based on the switch LID (from previous step 2).
* Matching the remote switch mst device representation to the relevant switch procedure:
Locate the switch in the "ibswitches" output and retrieve the switch LID. 
Convert the LID from Decimal to Hexadecimal base.
Locate the device with this LID (in hex base) from the "mst status" output. 

5) Invoke the following command, and provide the output:
flint -d <remote_switch_mst_device> q


We're looking forward to hearing from you.

Best regards,
Mellanox Technical Support

-------------------------------------------------
  Created by: Jubran Jubran 2016-12-01 16:41:35
Hello Vinícius Ferrão,

Thank you for contacting Mellanox Technical Support.

1) DId you try changing switch ports when testing performance between two FreeBSD machines through switch?
2) What happens when you try a back-to-back connection between two FreeBSD machines? Do you get good results?
3) Did you test the performance between FreeBSD and a Linux machine through switch?

Please, follow the provided procedure in order to query your unmanaged switch: 
 
1) Download and install MFT (Mellanox Firmware Tool):
http://www.mellanox.com/content/pages.php?pg=management_tools&menu_section=34
Refer to the User Manual for the installation instructions:
http://www.mellanox.com/pdf/MFT/MFT_user_manual.pdf
 
2) From one of the nodes in the fabric, invoke: ibswitches.
Match the relevant switch GUID from the Mellanox switch label to the LID.
 
3) From one of the nodes in the fabric , invoke: mst start -> mst ib add -> mst status
 
4) Identify the relevant unmanaged switch mst representation based on the switch LID (from previous step 2).
* Matching the remote switch mst device representation to the relevant switch procedure:
Locate the switch in the "ibswitches" output and retrieve the switch LID. 
Convert the LID from Decimal to Hexadecimal base.
Locate the device with this LID (in hex base) from the "mst status" output. 

5) Invoke the following command, and provide the output:
flint -d <remote_switch_mst_device> q


We're looking forward to hearing from you.

Best regards,
Mellanox Technical Support 
-------------------------------------------------------/


More information about the freebsd-infiniband mailing list