libthr shared locks

[Konstantin Belousov]

On Wed, Dec 23, 2015 at 01:22:16PM -0800, John Baldwin wrote:
> On Wednesday, December 23, 2015 10:18:37 PM Konstantin Belousov wrote:
> > On Wed, Dec 23, 2015 at 02:48:56PM -0500, Daniel Eischen wrote:
> > > On Wed, 23 Dec 2015, Konstantin Belousov wrote:
> > > 
> > > > On Wed, Dec 23, 2015 at 01:27:35PM -0500, Daniel Eischen wrote:
> > > >> On Wed, 23 Dec 2015, Konstantin Belousov wrote:
> > > >>
> > > >> [ ... ]
> > > >>> Would the ABI modified to make the pthread_mutex_t large enough to
> > > >>> hold struct pthread_mutex, the rest of the implementation of the
> > > >>> shared mutex is relatively trivial, if not already done.
> > > >>>
> > > >>> Changing this ABI is very hard.  libthr provides the symbol
> > > >>> versioning, which allows to provide compatible shims for the previous
> > > >>> ABI variant.  But since userspace tends to use the pthread objects in
> > > >>> the layouts of the library objects, this causes serious ABI issues
> > > >>> when mixing libraries built against different default versions of
> > > >>> libthr.
> > > >>
> > > >> I think this is only if the libraries (or apps) pass pthread
> > > >> lock types between them, that one has initialized with one ABI
> > > >> but the other library uses another ABI.  We should really
> > > >> exclude this as part of our ABI compatibility.
> > > > Applications commonly embed pthread locks into their objects, including
> > > > the exposed ABI in the third-party libraries.
> > > >
> > > > struct my_object {
> > > > 	pthread_mutex_t obj_lock;
> > > > 	...
> > > > };
> > > >
> > > > Changing the size of the pthread locks causes uncontrolled breakage of
> > > > the ABI for significant number of third-party code.
> > > 
> > > If the application creates the object itself or allocates storage
> > > for it, basically, if it isn't opaque, yes.  But we can bump port
> > > revisions for affected libraries (probably just searching
> > > /usr/local/include/... for pthread_mutex, pthread_cond, etc
> > > types to see possible problems.  I did a search for the installed
> > > ports on my system and found a few that might cause problems.
> > This relegates the issue to an attempt to do the full rebuild.  But I
> > do not see how the port bump would fix it, assume that you are updating
> > from the 10.x to 11.x and have the mix of the libraries, some of which
> > were built during the 10.x lifetime but with the bumped ports version.
> > 
> > It is not feasible to do a reliable audit of the 24+ Kports.
> 
> As a bit of a devil's advocate, I think the 64-bit ino_t change will in
> fact require this for 11.  I suspect 3rd pary apps embed struct stat in
> various structures as well and that that ABI change will require not
> mixing old and new libraries.
The stat change is different in this.  I do think (and quick overview
of includes seems to support the opinion) that the struct stat is rarely
embedded into the user objects.  I already wrote about this, when I said
that 'struct stat' changes are mostly confined to the src/ tree, in one
of the previous messages.  When I thought about the ino_t 64bit change,
the only example of affected app that I could remember, without looking 
at the code, was rogue/nethack.

> 
> One other point in favor of Konstantin's approach (IMO) is that keeping
> the structures private prevents having to maintain the ABI of those
> structures in the future.  I'm already keenly aware of how painful a
> problem that can be with our non-opaque FILE (and which we cannot now
> make opaque even though the standard APIs would work fine with an opaque
> object).

My patch does not prevent future change to use in-place pthread_mutex_t,
since the patch does not change current ABI. If somebody is willing to
go that route, well, fine. Nobody productized David' work for ten (?)
years. And the reason is, I believe, that the ABI change required to the
fundamental facility (threads did become ubiquios) at that late point in
the OS lifecycle is destructive.

The issue we have after the embedded structures layout modification is
exactly the issue which cannot be handled by symbol versioning.