A mutex for inter-process ;-)

[Randall Stewart]

On Mar 30, 2009, at 10:18 PM, David Xu wrote:

> Daniel Eischen wrote:
>> On Mon, 30 Mar 2009, Randall Stewart wrote:
>>>
>>> On Mar 30, 2009, at 5:22 PM, Daniel Eischen wrote:
>>>
>>>> On Mon, 30 Mar 2009, Randall Stewart wrote:
>>>>
>>>>> Hi all:
>>>>>
>>>>> I have recently written a small set of routines that allow
>>>>> two process to have a "mutex" between them.. actually it allows
>>>>> all of the threads in any set of processes to have mutexes  
>>>>> between themselves ;-)
>>>>>
>>>>> Anyway it seems to be working fairly well.. I still have to  
>>>>> write a man page
>>>>> for it (documentation always last).. and eventually I would like  
>>>>> to port in
>>>>> some of the WITNESS type features since the mutex's have names..
>>>>>
>>>>> I probably should also think about scaling it up a bit.. right  
>>>>> now its really
>>>>> more for a small scale (100 or less mutexes)...
>>>>>
>>>>> Who should I talk to about getting this in... having it reviewed  
>>>>> etc. I think
>>>>> it belongs in libthr since it really needs the tid of the  
>>>>> pthreads from the
>>>>> pthread_t type... and for now I have a horrible hack in to get  
>>>>> it ;-)
>>>>
>>>> The real way to do this is to support PTHREAD_PROCESS_SHARED
>>>> mutexes within our normal mutex, and to change our current
>>>> mutex (and cv) types to be structs instead of pointers.
>>>> The current API, other than the type change, shouldn't
>>>> change at all.
>>>
>>>
>>> So how do you propose to name the mutex's so that two disparate
>>> process can locate the same mutex?
>> They are placed in shared memory, according to POSIX.
>>> I don't see how a pthread_mutex can suffice... we need more than
>>> just the current mutex...
>>>
>>> What am I missing?
>> As far as I know, David Xu implemented the kernel hooks
>> for umtx (the underlying mutex in pthread mutex) to be
>> shared.  As soon as you can place the entire userland
>> pthread_mutex_t struct in shared memory, it should all
>> just work (with probably some trivial changes in libthr).
>> The harder part is versioning all the symbols that
>> currently think pthread_mutex_t, pthread_cond_t, etc,
>> are pointers, and defining the structs with enough
>> foresight so that it is unlikely we have to modify
>> them in the future (causing a future ABI breakage),
>> and also aligning them nicely for the various archs.
>> You should really look at how POSIX defines process
>> shared mutex, cvs, etc.  See:
>>  pthread_barrierattr_[gs]etpshared()
>>  pthread_condattr_[gs]etpshared()
>>  pthread_mutexattr_[gs]etpshared()
>>  pthread_wrlockattr_[gs]etsphared()
>> You can use this as a starting point:
>>  http://www.opengroup.org/onlinepubs/009695399/
>>  http://www.opengroup.org/onlinepubs/009695399/functions/pthread_barrierattr_setpshared.html 
>>    http://www.opengroup.org/onlinepubs/009695399/functions/pthread_condattr_setpshared.html 
>>    http://www.opengroup.org/onlinepubs/009695399/functions/pthread_mutexattr_setpshared.html 
>>    http://www.opengroup.org/onlinepubs/009695399/functions/pthread_rwlockattr_setpshared.html
>
> You are right. umtx is ready for process-shared mutex, condition
> variable and rwlock. We are blocked by our pthread_mutex_t
> and pthread_cond_t definitions which are pointers, mmap()ing it into
> shared memory and calling pthread API will not work correctly, they
> should be defined as a block of memory.

Yes, the stuff I have been playing with uses umtx... it works
seamlessly...

>
> Recent POSIX standard introduces robust mutex type which can detects
> mutex owner's death, but in theory, shared memory model will never  
> be robust.

I agree, but its something someone I interviewed with was asking  
about... and
they were busy going about making kernel hacks to add shared mutex's  
which
led me down the path of looking what's there and not there..

I will go poke around and look at the posix stuff...

I wonder if some sort of extensions in the kernel might be a good thing
to do with a shared memory model to deal with the "robustness" issue.  
Something
to think about for sure.

R

>
>
> Regards,
> David Xu
>

------------------------------
Randall Stewart
803-317-4952 (cell)
803-345-0391(direct)