Device hints to put mlx4 on IB mode

Meny Yossefi menyy at mellanox.com
Sun Nov 13 13:58:23 UTC 2016


Hi, 

There's a tool called 'mlxconfig' which can change port's type, but I think it requires the driver to be loaded, so no real help here.
You can try contacting our support team (support at mellanox.com) about the tool.

Which kernel version are you using? 
In recent versions, we added a patch the forces port type to come up as ETH. 
In that case, changing it to IB will just be a one liner to maintain. 


-Meny


-----Original Message-----
From: Vinícius Ferrão [mailto:ferrao at vmio.com.br] 
Sent: Thursday, November 10, 2016 5:40 PM
To: Meny Yossefi <menyy at mellanox.com>
Cc: freebsd-infiniband at freebsd.org; Mark Bloch <markb at mellanox.com>; Yuval Bason <yuvalba at mellanox.com>; Hans Petter Selasky <hanss at mellanox.com>
Subject: Re: Device hints to put mlx4 on IB mode

Hello Meny,

Thanks for the input. You’re recommending a driver code change to start always on IB mode. This appears to be OK, but it's little dangerous. I will need to keep the patches for the drivers/modules if anything changes, right?

Another question, changing the starting mode cannot be done in other place? Like the Connect-X3 firmware? If not, since you’re on Mellanox can we recommend a better approach from the upstream (mlx) or this would be inviable?

Thanks for your attention and time, it was really helpful.

V.

> On 10 Nov 2016, at 10:21, Meny Yossefi <menyy at mellanox.com> wrote:
> 
> Hi,
> 
> I'm not familiar of any hints passed from the bootloader.
> 
> We do have some FW tools to identify number of NIC ports, but I guess that won't serve your purpose.
> 
> The only option I can think of at the moment is to force all ports to come up in IB mode.
> 
> Try using this as reference: 
> 
> sys/ofed/drivers/net/mlx4/main.c:
> 
> 
> static struct param_data port_type_array = {
>        .dbdf2val = {
>                .name           = "port_type_array param",
>                .num_vals       = 2,
> -           .def_val        = {MLX4_PORT_TYPE_NONE, MLX4_PORT_TYPE_NONE},
> +         .def_val    = {MLX4_PORT_TYPE_IB, MLX4_PORT_TYPE_IB},
>                .range          = {MLX4_PORT_TYPE_IB, MLX4_PORT_TYPE_NA}
>        }
> };
> @@ -759,6 +759,9 @@ static int mlx4_dev_cap(struct mlx4_dev *dev, struct mlx4_dev_cap *dev_cap)
>                         (dev->caps.flags & MLX4_DEV_CAP_FLAG_DPDP) &&
>                         (dev->caps.flags & 
> MLX4_DEV_CAP_FLAG_SENSE_SUPPORT));
> 
> +         /* MY: disablling auto sense for default IB/Eth ports support */
> +         mlx4_priv(dev)->sense.sense_allowed[i] = 0;
> +
>                /*
>                 * If "default_sense" bit is set, we move the port to "AUTO" mode
>                 * and perform sense_port FW command to try and set the 
> correct
> 
> -Meny
> 
> 
> -----Original Message-----
> From: freebsd-commits-tracker
> Sent: Wednesday, November 09, 2016 1:55 PM
> To: Mark Bloch <markb at mellanox.com>; Meny Yossefi 
> <menyy at mellanox.com>; Yuval Bason <yuvalba at mellanox.com>
> Subject: FW: Device hints to put mlx4 on IB mode
> 
> 
> 
> 
> ________________________________________
> From: owner-freebsd-infiniband at freebsd.orgOn Behalf OfVinícius Ferrão
> Sent: Tuesday, November 8, 2016 4:27:13 PM (UTC+00:00) Monrovia, 
> Reykjavik
> To: freebsd-infiniband at freebsd.org
> Subject: Device hints to put mlx4 on IB mode
> 
> Hello guys,
> 
> I would like to put my Connect-X3 card in IB mode during boot time using device hints instead of sysctl's.
> 
> Today we have this dirty workarround on /etc/sysctl.conf:
> sys.device.mlx4_core0.mlx4_port1=ib
> sys.device.mlx4_core0.mlx4_port2=ib
> 
> It's fine when we use a Connect-X3 card with two Infiniband interfaces, but if a ConnectX-3 card with only one interface is used an error message is dumped on the console about failing to set the "sys.device.mlx4_core0.mlx4_port2=ib" sysctl.
> 
> To solve this I would like to use device hints passed from the GRUB2 bootloader, but I was unable to find which hints I should use for the mlx4 drivers. I don't even know if this is supported.
> 
> The problem is purely cosmetic, I just want to get rid of the error message during the boot process.
> 
> Thanks in advance,
> V.
> 
> 
> 
> 
> _______________________________________________
> freebsd-infiniband at freebsd.org mailing list 
> https://lists.freebsd.org/mailman/listinfo/freebsd-infiniband
> To unsubscribe, send any mail to "freebsd-infiniband-unsubscribe at freebsd.org"
> _______________________________________________
> freebsd-infiniband at freebsd.org mailing list 
> https://lists.freebsd.org/mailman/listinfo/freebsd-infiniband
> To unsubscribe, send any mail to "freebsd-infiniband-unsubscribe at freebsd.org"



More information about the freebsd-infiniband mailing list