Stuck CLOSED sockets / sshd / zombies...

[Konstantin Belousov]

On Tue, Apr 08, 2014 at 10:06:36PM +0100, Karl Pielorz wrote:
> 
> 
> --On 8 April 2014 15:33:53 -0400 John Baldwin <jhb at freebsd.org> wrote:
> 
> > I think this patch only changes debug info, so I think you can just build
> > and install and re-attach gdb without having to restart sshd.  Also, this
> > patch should only require you to build and install libexec/rtld-elf.
> 
> Ok, patched, re-attached and re-ran bt (output is below). I've got gdb from 
> ports building on that machine but I probably won't be able to do any more 
> with it tonight.
> 
> -Karl
> 
> ---
> 
> [Switching to LWP 100218]
> 0x00000008038ea89c in __error () from /lib/libthr.so.3
> (gdb) bt
> #0  0x00000008038ea89c in __error () from /lib/libthr.so.3
> #1  0x00000008038e104f in __thr_rwlock_rdlock (rwlock=0x803af9480, 
> flags=<value optimized out>, tsp=<value optimized out>)
>     at /usr/src/lib/libthr/thread/thr_umtx.c:277
> #2  0x00000008038e821c in _thr_rtld_rlock_acquire (lock=0x803af9480) at 
> atomic.h:143
> #3  0x000000080064f9a2 in _rtld_get_stack_prot () from /libexec/ld-elf.so.1
> #4  0x00000008006498c9 in r_debug_state () from /libexec/ld-elf.so.1
> #5  0x00000008006470cd in .text () from /libexec/ld-elf.so.1
> #6  0x000000000041072c in grace_alarm_handler (sig=-17504) at 
> /usr/src/secure/usr.sbin/sshd/../../../crypto/openssh/sshd.c:378
> #7  <signal handler called>
> #8  0x00000008038ea89c in __error () from /lib/libthr.so.3
> #9  0x00000008038e104f in __thr_rwlock_rdlock (rwlock=0x803af9480, 
> flags=<value optimized out>, tsp=<value optimized out>)
>     at /usr/src/lib/libthr/thread/thr_umtx.c:277
> #10 0x00000008038e821c in _thr_rtld_rlock_acquire (lock=0x803af9480) at 
> atomic.h:143
> #11 0x000000080064f9a2 in _rtld_get_stack_prot () from /libexec/ld-elf.so.1
> #12 0x00000008006498c9 in r_debug_state () from /libexec/ld-elf.so.1
> #13 0x00000008006470cd in .text () from /libexec/ld-elf.so.1
> #14 0x000000000042f9dd in sshpam_sigchld_handler (sig=<value optimized 
> out>) at 
> /usr/src/secure/usr.sbin/sshd/../../../crypto/openssh/auth-pam.c:152
> #15 <signal handler called>
> #16 0x0000000800653aea in _rtld_atfork_post () from /libexec/ld-elf.so.1
> #17 0x000000080064a1eb in dlclose () from /libexec/ld-elf.so.1
> #18 0x0000000800edd121 in openpam_clear_chains () from /usr/lib/libpam.so.5
> #19 0x0000000800edd0bc in openpam_clear_chains () from /usr/lib/libpam.so.5
> #20 0x0000000800edd0bc in openpam_clear_chains () from /usr/lib/libpam.so.5
> #21 0x0000000800edd061 in openpam_clear_chains () from /usr/lib/libpam.so.5
> #22 0x0000000800ed99e7 in pam_end () from /usr/lib/libpam.so.5
> #23 0x000000000042e15d in sshpam_cleanup () at 
> /usr/src/secure/usr.sbin/sshd/../../../crypto/openssh/auth-pam.c:614
> #24 0x000000000041d58f in do_cleanup (authctxt=0x80401a600) at 
> /usr/src/secure/usr.sbin/sshd/../../../crypto/openssh/session.c:2732
> #25 0x000000000041064f in ssh_cleanup_exit (i=255) at 
> /usr/src/secure/usr.sbin/sshd/../../../crypto/openssh/sshd.c:2545
> #26 0x0000000000428f83 in mm_request_receive (sock=<value optimized out>, 
> m=<value optimized out>)
>     at 
> /usr/src/secure/usr.sbin/sshd/../../../crypto/openssh/monitor_wrap.c:153
> #27 0x0000000000427e26 in monitor_read (pmonitor=0x804022220, 
> ent=0x6465a0, pent=0x7fffffffd0c0)
>     at /usr/src/secure/usr.sbin/sshd/../../../crypto/openssh/monitor.c:593
> #28 0x0000000000427b49 in monitor_child_preauth (_authctxt=<value 
> optimized out>, pmonitor=0x804022220)
>     at /usr/src/secure/usr.sbin/sshd/../../../crypto/openssh/monitor.c:387
> #29 0x000000000040fd15 in main (ac=<value optimized out>, av=<value 
> optimized out>) at 
> /usr/src/secure/usr.sbin/sshd/../../../crypto/openssh/sshd.c:679

Ok, so the patch sort of worked, but your rtld does not have debugging
information for compiled .c files.  Please, in the patched tree, do
the following:

cd libexec/rtld-elf
make clean
make DEBUG_FLAGS=-g DEBUG=-DDEBUG
make install

and then re-do the backtracing.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 834 bytes
Desc: not available
URL: <http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20140409/dcc6b51a/attachment.sig>