A mutex for inter-process ;-)

[David Xu]

Daniel Eischen wrote:
> On Mon, 30 Mar 2009, Randall Stewart wrote:
> 
>>
>> On Mar 30, 2009, at 5:22 PM, Daniel Eischen wrote:
>>
>>> On Mon, 30 Mar 2009, Randall Stewart wrote:
>>>
>>>> Hi all:
>>>>
>>>> I have recently written a small set of routines that allow
>>>> two process to have a "mutex" between them.. actually it allows
>>>> all of the threads in any set of processes to have mutexes between 
>>>> themselves ;-)
>>>>
>>>> Anyway it seems to be working fairly well.. I still have to write a 
>>>> man page
>>>> for it (documentation always last).. and eventually I would like to 
>>>> port in
>>>> some of the WITNESS type features since the mutex's have names..
>>>>
>>>> I probably should also think about scaling it up a bit.. right now 
>>>> its really
>>>> more for a small scale (100 or less mutexes)...
>>>>
>>>> Who should I talk to about getting this in... having it reviewed 
>>>> etc. I think
>>>> it belongs in libthr since it really needs the tid of the pthreads 
>>>> from the
>>>> pthread_t type... and for now I have a horrible hack in to get it ;-)
>>>
>>> The real way to do this is to support PTHREAD_PROCESS_SHARED
>>> mutexes within our normal mutex, and to change our current
>>> mutex (and cv) types to be structs instead of pointers.
>>> The current API, other than the type change, shouldn't
>>> change at all.
>>
>>
>> So how do you propose to name the mutex's so that two disparate
>> process can locate the same mutex?
> 
> They are placed in shared memory, according to POSIX.
> 
>> I don't see how a pthread_mutex can suffice... we need more than
>> just the current mutex...
>>
>> What am I missing?
> 
> As far as I know, David Xu implemented the kernel hooks
> for umtx (the underlying mutex in pthread mutex) to be
> shared.  As soon as you can place the entire userland
> pthread_mutex_t struct in shared memory, it should all
> just work (with probably some trivial changes in libthr).
> The harder part is versioning all the symbols that
> currently think pthread_mutex_t, pthread_cond_t, etc,
> are pointers, and defining the structs with enough
> foresight so that it is unlikely we have to modify
> them in the future (causing a future ABI breakage),
> and also aligning them nicely for the various archs.
> 
> You should really look at how POSIX defines process
> shared mutex, cvs, etc.  See:
> 
>   pthread_barrierattr_[gs]etpshared()
>   pthread_condattr_[gs]etpshared()
>   pthread_mutexattr_[gs]etpshared()
>   pthread_wrlockattr_[gs]etsphared()
> 
> You can use this as a starting point:
> 
>   http://www.opengroup.org/onlinepubs/009695399/
>   
> http://www.opengroup.org/onlinepubs/009695399/functions/pthread_barrierattr_setpshared.html 
> 
>   
> http://www.opengroup.org/onlinepubs/009695399/functions/pthread_condattr_setpshared.html 
> 
>   
> http://www.opengroup.org/onlinepubs/009695399/functions/pthread_mutexattr_setpshared.html 
> 
>   
> http://www.opengroup.org/onlinepubs/009695399/functions/pthread_rwlockattr_setpshared.html 
> 
> 

You are right. umtx is ready for process-shared mutex, condition
variable and rwlock. We are blocked by our pthread_mutex_t
and pthread_cond_t definitions which are pointers, mmap()ing it into
shared memory and calling pthread API will not work correctly, they
should be defined as a block of memory.
Recent POSIX standard introduces robust mutex type which can detects
mutex owner's death, but in theory, shared memory model will never be 
robust.

Regards,
David Xu