nginx + passenger = segv in _rtld_error on restart on FreeBSD8.0?

[Matt Reimer]

On Wed, Dec 9, 2009 at 4:20 AM, Steven Hartland <killing at multiplay.co.uk> wrote:
>
> ----- Original Message ----- From: "Kostik Belousov" <kostikbel at gmail.com>
> To: "Steven Hartland" <killing at multiplay.co.uk>
> Cc: <freebsd-hackers at freebsd.org>; <freebsd-stable at freebsd.org>
> Sent: Wednesday, December 09, 2009 10:21 AM
> Subject: Re: nginx + passenger = segv in _rtld_error on restart on
> FreeBSD8.0?
>
> This is the trace once world had been recompiled with:-
> CFLAGS=-pipe
> WITH_CTF=1
> DEBUG_FLAGS=-g
>
>
> #0  0x0000000800c95eec in thr_kill () at thr_kill.S:3
> #1  0x0000000800b22e9e in _thr_send_sig (thread=0x800f06600, sig=6) at
> /usr/src/lib/libthr/thread/thr_kern.c:92
> #2  0x0000000800b1f878 in _raise (sig=6) at
> /usr/src/lib/libthr/thread/thr_sig.c:187
> #3  0x0000000800d74003 in abort () at /usr/src/lib/libc/stdlib/abort.c:65
> #4  0x000000000043b8a7 in Client::threadMain (this=0x800f9cf40) at
> ext/nginx/HelperServer.cpp:516
> #5  0x0000000000411302 in boost::_mfi::mf0<void, Client>::operator()
> (this=0x7fffffa45ea8, p=0x800f9cf40) at mem_fn_template.hpp:49
> #6  0x0000000000411651 in boost::_bi::list1<boost::_bi::value<Client*>
>>::operator()<boost::_mfi::mf0<void, Client>, boost::_bi::list0>
> (this=0x7fffffa45eb8, f=@0x7fffffa45ea8, a=@0x7fffffa45d7f) at bind.hpp:232
> #7  0x0000000000411696 in boost::_bi::bind_t<void, boost::_mfi::mf0<void,
> Client>, boost::_bi::list1<boost::_bi::value<Client*> > >::operator()
> (this=0x7fffffa45ea8) at bind_template.hpp:20
> #8  0x00000000004116bd in
> boost::detail::function::void_function_obj_invoker0<boost::_bi::bind_t<void,
> boost::_mfi::mf0<void, Client>, boost::_bi::list1<boost::_bi::value<Client*>
>> >, void>::invoke (
>   function_obj_ptr=@0x7fffffa45ea8) at function_template.hpp:158
> #9  0x000000000042e73a in boost::function0<void, std::allocator<void>
>>::operator() (this=0x7fffffa45ea0) at function_template.hpp:825
> #10 0x0000000000435760 in oxt::thread::thread_main (func=@0x7fffffa45ea0,
> data=@0x7fffffa45e90) at thread.hpp:107
> #11 0x000000000041310e in
> boost::_bi::list2<boost::_bi::value<boost::function<void ()(),
> std::allocator<void> > >,
> boost::_bi::value<boost::shared_ptr<oxt::thread::thread_data> >
>>::operator()<void (*)(boost::function<void ()(), std::allocator<void> >,
> boost::shared_ptr<oxt::thread::thread_data>), boost::_bi::list0>
> (this=0x800f3ee80, f=@0x800f3ee78, a=@0x7fffffa45f0f) at bind.hpp:289
> #12 0x0000000000413196 in boost::_bi::bind_t<void, void
> (*)(boost::function<void ()(), std::allocator<void> >,
> boost::shared_ptr<oxt::thread::thread_data>),
> boost::_bi::list2<boost::_bi::value<boost::function<void ()(),
> std::allocator<void> > >,
> boost::_bi::value<boost::shared_ptr<oxt::thread::thread_data> > >
>>::operator() (this=0x800f3ee78) at bind_template.hpp:20
> #13 0x00000000004131b9 in
> boost::thread::thread_data<boost::_bi::bind_t<void, void
> (*)(boost::function<void ()(), std::allocator<void> >,
> boost::shared_ptr<oxt::thread::thread_data>),
> boost::_bi::list2<boost::_bi::value<boost::function<void ()(),
> std::allocator<void> > >,
> boost::_bi::value<boost::shared_ptr<oxt::thread::thread_data> > > > >::run
> (this=0x800f3ee00) at thread.hpp:130
> #14 0x0000000000443259 in thread_proxy (param=0x800f3ee00) at
> ext/boost/src/pthread/thread.cpp:127
> #15 0x0000000800b1badd in thread_start (curthread=0x800f06600) at
> /usr/src/lib/libthr/thread/thr_create.c:288
> #16 0x0000000000000000 in ?? ()
> Cannot access memory at address 0x7fffffa46000
> Current language:  auto; currently asm
>
> It seems that in the passenger client threads it calls closeStream which
> errors when
> the socket close errors with ENOTCONN
>       virtual void closeStream() {
>           TRACE_POINT();
>           if (fd != -1) {
>               int ret = syscalls::close(fd);
>               fd = -1;
>               if (ret == -1) {
>                   if (errno == EIO) {
>                       throw SystemException("A write operation on the
> session stream failed",
>                           errno);
>                   } else {
>                       throw SystemException("Cannot close the session
> stream",
>                           errno);
>                   }
>               }
>           }
>       }
>
> This causes it to call abort on the the thread which then crashes the app
> with
> the above stack trace, which seems really weird. Anyone got any ideas?
>
>   Regards
>   steve

Steve,

The patch for PR 144061 works for us.

http://lists.freebsd.org/pipermail/freebsd-hackers/2010-February/030741.html
http://www.freebsd.org/cgi/query-pr.cgi?pr=144061

Matt