[PATCH]: ipoib with mlx4 initialisation ordering

[Andreas Kempe]

On Wed, Feb 26, 2020 at 10:05:55PM +0100, Andreas Kempe wrote:
> On Mon, Feb 24, 2020 at 11:50:10PM +0100, Hans Petter Selasky wrote:
> > Hi,
> > 
> > On 2020-02-24 20:46, Andreas Kempe wrote:
> > > If you want me to try reordering the deinitialisation, I should be
> > > able to do that this coming wednesday.
> > > 
> > 
> > Yes, please.
> > 
> > Depending on how lists work, might not be so good in the long run.
> > 
> 
> First I tried the following change and the machine still hung.
> 
> > --- sys/ofed/drivers/infiniband/ulp/ipoib/ipoib_main.c	(revision 356611)
> > +++ sys/ofed/drivers/infiniband/ulp/ipoib/ipoib_main.c	(working copy)
> > @@ -1739,7 +1739,7 @@
> >  }
> >  
> >  module_init(ipoib_init_module);
> > -module_exit(ipoib_cleanup_module);
> > +module_exit_order(ipoib_cleanup_module, SI_ORDER_FIRST);
> >  
> >  static int
> >  ipoib_evhand(module_t mod, int event, void *arg)
> 
> Then I tried moving the mlx4 driver unloading using the following
> change and the machine still hung.
> 
> > --- sys/dev/mlx4/mlx4_ib/mlx4_ib_main.c	(revision 356611)
> > +++ sys/dev/mlx4/mlx4_ib/mlx4_ib_main.c	(working copy)
> > @@ -3320,7 +3320,7 @@
> >  }
> >  
> >  module_init_order(mlx4_ib_init, SI_ORDER_THIRD);
> > -module_exit(mlx4_ib_cleanup);
> > +module_exit_order(mlx4_ib_cleanup, SI_ORDER_THIRD);
> >  
> >  static int
> >  mlx4ib_evhand(module_t mod, int event, void *arg)
> 
> I don't really feel like analysing why it still hangs with the above
> changes at the moment since we got something that works for us. If you
> have any suggestions you want me to try, I could still do that.
> 
> > > > I'm currently trying to reproduce the issue.
> > 
> > > > 
> > > We're seeing the issue every time when running the machine in a
> > > network with a Linux machine. We simply need to send a bit of data on
> > > the link and then trigger a shutdown.
> > 
> > I see.
> > 
> 
> I can add that we compiled the modules into the kernel by adding the
> following to the GENERIC kernel of 12.1-STABLE:
> 
> > # INFINIBAND
> > options		COMPAT_LINUXKPI
> > options		OFED
> > options		SDP
> > options		IPOIB_CM
> > 
> > device		ipoib
> > device		mlx4
> > device		mlx4ib
> 
> Adding freebsd-inifinband again.
> 
> Cordially,
> Andreas Kempe


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://lists.freebsd.org/pipermail/freebsd-infiniband/attachments/20200226/13c81104/attachment.sig>