Changing Between Infiniband Connected Mode and Datagram Mode at Runtime on FreeBSD 12.2

Pokala, Ravi rpokala at
Thu Mar 4 19:00:47 UTC 2021

-----Original Message-----
From: Hans Petter Selasky <hps at>
Date: 2021-03-04, Thursday at 02:13
To: "Foster, Greg" <gfoster at>, "freebsd-infiniband at" <freebsd-infiniband at>
Cc: Ravi Pokala <rpokala at>
Subject: Re: Changing Between Infiniband Connected Mode and Datagram Mode at Runtime on FreeBSD 12.2

    On 3/4/21 3:49 AM, Foster, Greg wrote:
    > Hello FreeBSD IB,
    > What is the procedure to change between Infiniband Connected Mode and Datagram Mode
    > at runtime on FreeBSD 12.2?
    > I've seen different references on the WEB, but the procedure still seems unclear.
    > In one reference
    > refers to the Linux procedure, which is not applicable on FreeBSD.
    > Another reference,, implies the driver must be recompiled
    > to change modes.
    > Yet another,
    > Suggests it can be done at runtime to add a tuneable/sysctl but doesn't specify any details...
    > Is the solution to maybe compile the drivers twice, once for CM and once for DM, keeping both around
    > and loading the desired one?


    For IPoIB there is an option you need to set at compile time for this 
    feature to be activated:

    sys/conf/options:IPOIB_CM	opt_ofed.h


Hi Hans,

It looks like IPOIB_CM causes CONFIG_INFINIBAND_IPOIB_CM to be defined. I see that CONFIG_INFINIBAND_IPOIB_CM adds fields to structures, which is why it can't be a run-time or boot-time tunable. :-(

But it looks like the changes associated with CONFIG_INFINIBAND_IPOIB_CM are all additions, not replacing datagram code with connected-mode code. Other than having unnecessary fields in structures, would there be any harm in building with IPOIB_CM, but running in datagram mode? Or are you saying that we first need to build with IPOIB_CM, and then we can use a tunable to switch modes?



More information about the freebsd-infiniband mailing list