ib_unregister_device - OFED related question

[Somayajulu, David]

Hi All,

ib_unregister_device<http://fxr.watson.org/fxr/ident?v=FREEBSD10;i=ib_unregister_device>() is typically called as part of the rdma driver module unload for a given device. As part of execution of this function it loops through the following code:



ofed/drivers/inifiniband/core/device.c



void ib_unregister_device<http://fxr.watson.org/fxr/ident?v=FREEBSD10;i=ib_unregister_device>(struct ib_device<http://fxr.watson.org/fxr/ident?v=FREEBSD10;i=ib_device> *device<http://fxr.watson.org/fxr/ident?v=FREEBSD10;i=device>)

{

                    :

                    list_for_each_entry_reverse<http://fxr.watson.org/fxr/ident?v=FREEBSD10;i=list_for_each_entry_reverse>(client<http://fxr.watson.org/fxr/ident?v=FREEBSD10;i=client>, &client_list, list<http://fxr.watson.org/fxr/ident?v=FREEBSD10;i=list>)

                                         if (client<http://fxr.watson.org/fxr/ident?v=FREEBSD10;i=client>->remove)

                                                             client<http://fxr.watson.org/fxr/ident?v=FREEBSD10;i=client>->remove(device<http://fxr.watson.org/fxr/ident?v=FREEBSD10;i=device>);

                    :

}



For IB clients, client->remove(device) calls cma_remove_one() (ofed/drivers/inifiniband/core/cma.c) which ends up calling cma_process_remove().



cma_process_remove() calls wait_for_completion(&cma_dev->comp).



Now wait_for_completion() can potentially cause the thread to sleep().



>From kern/kern_module.c:

int

module_unload<http://fxr.watson.org/fxr/ident?v=FREEBSD10;i=module_unload>(module_t<http://fxr.watson.org/fxr/ident?v=FREEBSD10;i=module_t> mod<http://fxr.watson.org/fxr/ident?v=FREEBSD10;i=mod>)

{

         int error<http://fxr.watson.org/fxr/ident?v=FREEBSD10;i=error>;



         mtx_lock<http://fxr.watson.org/fxr/ident?v=FREEBSD10;i=mtx_lock>(&Giant<http://fxr.watson.org/fxr/ident?v=FREEBSD10;i=Giant>);

         error<http://fxr.watson.org/fxr/ident?v=FREEBSD10;i=error> = MOD_EVENT<http://fxr.watson.org/fxr/ident?v=FREEBSD10;i=MOD_EVENT>(mod<http://fxr.watson.org/fxr/ident?v=FREEBSD10;i=mod>, MOD_UNLOAD<http://fxr.watson.org/fxr/ident?v=FREEBSD10;i=MOD_UNLOAD>);

         mtx_unlock<http://fxr.watson.org/fxr/ident?v=FREEBSD10;i=mtx_unlock>(&Giant<http://fxr.watson.org/fxr/ident?v=FREEBSD10;i=Giant>);

         return (error<http://fxr.watson.org/fxr/ident?v=FREEBSD10;i=error>);

}

We see that module_unload() grabs the Gaint Lock prior to invoking UNLOAD. Isn't this a problem with cma_process_remove() or am I missing something?



Thanks

David S. (davidcs at freebsd.org<mailto:davidcs at freebsd.org>)