Stuck CLOSED sockets / sshd / zombies...

[Konstantin Belousov]

On Fri, Apr 11, 2014 at 03:57:00PM +0100, Karl Pielorz wrote:
> 
> 
> --On 11 April 2014 17:15 +0300 Konstantin Belousov <kostikbel at gmail.com> 
> wrote:
> 
> > So my suspicious idea seems to be true. From the ldd output, libc
> > appears before libthr in the global order, so libc sigaction() symbol
> > is resolved before libthr interposer. The result is that libthr wrapper
> > thr_sighandler() for the signal handlers is not installed as the
> > recepient of the kernel signal, which prevents libthr locks for rtld
> > from working properly.
> 
> Ok, I can just about follow that ;)
> 
> > To confirm or deny my theory, please apply the patch below, in addition to
> > the previous patch, and rebuild sshd only,
> ># cd src/secure/usr.sbin/sshd && make clean all install
> > The patch tilts the order of initialization, for my build I got
> > sandy% ldd /usr/sbin/sshd
> > ...
> >         libz.so.6 => /lib/libz.so.6 (0x802f0d000)
> >         libthr.so.3 => /lib/libthr.so.3 (0x803123000)
> >         libc.so.7 => /lib/libc.so.7 (0x803348000)
> >         libldns.so.5 => /usr/lib/private/libldns.so.5 (0x8036d1000)
> >         libmd.so.6 => /lib/libmd.so.6 (0x803926000)
> > which could be enough to prevent the bug.
> >
> > Please retest and report.
> 
> Ok, patched the makefile - rebuilt / installed sshd restarted (which has 
> the same initialisation order as yours above), it and ran the security scan 
> against it.
> 
> *This does indeed seem to fix the problem*
> 
> The scan completes, and there are no stuck 'urdlck' sshd's - and no socket 
> sitting around in CLOSE_WAIT or CLOSED - thanks! :)
> 
> I re-ran the scan a couple of times more to be sure, with the same result - 
> no zombies or anything.
Great.

> 
> Is this situation likely to be repeated anywhere else on the system? Or is 
> it likely just to be specific to sshd?
Well.

The issue with libthr so relying on interposition of libc has already bitten
us more than once.  The biggest practical consequence of it is that libthr
cannot be dynamically loaded, it must be linked to the main binary for the
whole construct to work.  This means that any program big enough to load
plugins at runtime must link to libthr if it might need to load plugin
linked to libthr.  Recently, perl and other programs from ports started
doing just that.

But this is first time I see interposing so broken due to wrong linking
order, esp. in the base system.

The correct solution is to merge libthr into libc. Some neccessary
preparations were already done, but the main work did not started yet.
This is huge efforts, and it probably should be coordinated with some
other ABI changes planed for libthr to support process-shared locks.

Anyway, for now, this patch, possibly enhanced to only link with
libpthread when kerberos is used, should be good enough.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 834 bytes
Desc: not available
URL: <http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20140411/71712c87/attachment.sig>