Stuck CLOSED sockets / sshd / zombies...

[Konstantin Belousov]

On Fri, Apr 11, 2014 at 01:39:54PM +0100, Karl Pielorz wrote:
> 
> Ok, rebuilt a debug world (with your rtld-elf patch), installed it - 
> reproduced the issue, and ran up gdb on a 'urdlck' stuck sshd, and got the 
> trace below.
The trace looks reasonable.

I vaguelly remember that you already answered this, but I want to start
investigating from the different angle.  Please show me the output
of 'ldd /usr/sbin/sshd' on your machine.  This happens on stable/10,
right ?

I do not see any linking with libpthread in the sshd Makefile.  Could it
be that libthr is loaded as dependency of some pam module ?

> 
> Fingers crossed,
> 
> -Karl
> 
> ps. When running up gdb I get a number of these errors (having checked, 
> I've always got these - I just didn't notice before as they scroll past 
> right at the top of the output from gdb starting up):
> 
> "
> Attaching to program: /usr/sbin/sshd, process 2220
> 
> warning: current_sos: Can't read pathname for load map: Bad address
> 
> 
> warning: current_sos: Can't read pathname for load map: Bad address
> "
> 
> I'm presuming they can be ignored?
Ignore this.

> 
> ---
> 
> "
> [Switching to Thread 804006400 (LWP 100083/sshd)]
> _umtx_op_err () at /usr/src/lib/libthr/arch/amd64/amd64/_umtx_op_err.S:37
> 37      RSYSCALL_ERR(_umtx_op)
> (gdb) bt
> #0  _umtx_op_err () at 
> /usr/src/lib/libthr/arch/amd64/amd64/_umtx_op_err.S:37
> #1  0x00000008038e304f in __thr_rwlock_rdlock (rwlock=0x803afb480, 
> flags=<value optimized out>, tsp=<value optimized out>)
>     at /usr/src/lib/libthr/thread/thr_umtx.c:277
> #2  0x00000008038ea22c in _thr_rtld_rlock_acquire (lock=0x803afb480) at 
> thr_umtx.h:196
> #3  0x000000080064f9a2 in rlock_acquire (lock=0x80085fe00, 
> lockstate=0x7fffffffc058) at /usr/src/libexec/rtld-elf/rtld_lock.c:197
> #4  0x00000008006498c9 in _rtld_bind (obj=0x800662000, reloff=13008) at 
> /usr/src/libexec/rtld-elf/rtld.c:675
> #5  0x00000008006470cd in _rtld_bind_start () at 
> /usr/src/libexec/rtld-elf/amd64/rtld_start.S:121
> #6  0x000000000041072c in grace_alarm_handler (sig=0) at 
> /usr/src/secure/usr.sbin/sshd/../../../crypto/openssh/sshd.c:378
> #7  <signal handler called>
> #8  _umtx_op_err () at 
> /usr/src/lib/libthr/arch/amd64/amd64/_umtx_op_err.S:37
> #9  0x00000008038e304f in __thr_rwlock_rdlock (rwlock=0x803afb480, 
> flags=<value optimized out>, tsp=<value optimized out>)
>     at /usr/src/lib/libthr/thread/thr_umtx.c:277
> #10 0x00000008038ea22c in _thr_rtld_rlock_acquire (lock=0x803afb480) at 
> thr_umtx.h:196
> #11 0x000000080064f9a2 in rlock_acquire (lock=0x80085fe00, 
> lockstate=0x7fffffffc668) at /usr/src/libexec/rtld-elf/rtld_lock.c:197
> #12 0x00000008006498c9 in _rtld_bind (obj=0x800662000, reloff=9240) at 
> /usr/src/libexec/rtld-elf/rtld.c:675
> #13 0x00000008006470cd in _rtld_bind_start () at 
> /usr/src/libexec/rtld-elf/amd64/rtld_start.S:121
> #14 0x000000000042f9dd in sshpam_sigchld_handler (sig=<value optimized 
> out>) at 
> /usr/src/secure/usr.sbin/sshd/../../../crypto/openssh/auth-pam.c:152
> #15 <signal handler called>
> #16 0x000000080064a1c4 in dlclose (handle=0x800696c00) at 
> /usr/src/libexec/rtld-elf/rtld.c:4136
> #17 0x0000000800ede121 in openpam_destroy_chain (chain=0x8040634e0)
>     at 
> /usr/src/lib/libpam/libpam/../../../contrib/openpam/lib/libpam/openpam_load.c:92
> #18 0x0000000800ede0bc in openpam_destroy_chain (chain=0x804063460)
>     at 
> /usr/src/lib/libpam/libpam/../../../contrib/openpam/lib/libpam/openpam_load.c:109
> #19 0x0000000800ede0bc in openpam_destroy_chain (chain=0x8040633e0)
>     at 
> /usr/src/lib/libpam/libpam/../../../contrib/openpam/lib/libpam/openpam_load.c:109
> #20 0x0000000800ede051 in openpam_clear_chains (policy=0x80401a6c8)
>     at 
> /usr/src/lib/libpam/libpam/../../../contrib/openpam/lib/libpam/openpam_load.c:128
> #21 0x0000000800eda9e7 in pam_end (pamh=0x80401a6c0, status=<value 
> optimized out>)
>     at 
> /usr/src/lib/libpam/libpam/../../../contrib/openpam/lib/libpam/pam_end.c:83
> #22 0x000000000042e15d in sshpam_cleanup () at 
> /usr/src/secure/usr.sbin/sshd/../../../crypto/openssh/auth-pam.c:614
> #23 0x000000000041d58f in do_cleanup (authctxt=0x80401a600) at 
> /usr/src/secure/usr.sbin/sshd/../../../crypto/openssh/session.c:2732
> #24 0x000000000041064f in ssh_cleanup_exit (i=255) at 
> /usr/src/secure/usr.sbin/sshd/../../../crypto/openssh/sshd.c:2545
> #25 0x0000000000428f83 in mm_request_receive (sock=<value optimized out>, 
> m=<value optimized out>)
>     at 
> /usr/src/secure/usr.sbin/sshd/../../../crypto/openssh/monitor_wrap.c:153
> #26 0x0000000000427e26 in monitor_read (pmonitor=0x804022220, ent=0x6465a0, 
> pent=0x7fffffffd240)
>     at /usr/src/secure/usr.sbin/sshd/../../../crypto/openssh/monitor.c:593
> #27 0x0000000000427b49 in monitor_child_preauth (_authctxt=<value optimized 
> out>, pmonitor=0x804022220)
>     at /usr/src/secure/usr.sbin/sshd/../../../crypto/openssh/monitor.c:387
> #28 0x000000000040fd15 in main (ac=<value optimized out>, av=<value 
> optimized out>)
>     at /usr/src/secure/usr.sbin/sshd/../../../crypto/openssh/sshd.c:679
> "
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 834 bytes
Desc: not available
URL: <http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20140411/9249527a/attachment.sig>