kern/131597: [kernel] c++ exceptions very slow on FreeBSD 7.1/amd64

[David Xu]

The following reply was made to PR kern/131597; it has been noted by GNATS.

From: David Xu <davidxu at freebsd.org>
To: John Baldwin <jhb at freebsd.org>
Cc: Kostik Belousov <kostikbel at gmail.com>, bug-followup at freebsd.org,
        guillaume at morinfr.org, kan at freebsd.org
Subject: Re: kern/131597: [kernel] c++ exceptions very slow on FreeBSD 7.1/amd64
Date: Fri, 27 Aug 2010 16:20:03 +0000

 John Baldwin wrote:
 > On Friday, April 23, 2010 10:41:11 am Kostik Belousov wrote:
 >> On Fri, Apr 23, 2010 at 10:21:41AM -0400, John Baldwin wrote:
 >>> On Friday 23 April 2010 9:47:40 am Kostik Belousov wrote:
 >>>> On Fri, Apr 23, 2010 at 08:43:41AM -0400, John Baldwin wrote:
 >>>>> On Friday 23 April 2010 8:25:01 am Kostik Belousov wrote:
 >>>>>> On Thu, Apr 22, 2010 at 04:09:34PM -0400, John Baldwin wrote:
 >>>>>>> I tracked the sigprocmask() system calls down to the operations to
 >>>>>>> acquire a write lock in the runtime linker. The lock was added to fix
 >>>>>>> an earlier bug with throwing exceptions in multithreaded C++ apps. The
 >>>>>>> relevant commit that added the lock is this:
 >>>>>>>
 >>>>>>>    http://svn.freebsd.org/viewvc/base?view=revision&revision=178807
 >>>>>>>
 >>>>>>> Are exceptions permitted during a signal handler? If not, then in
 >>>>>>> theory we would not need to invoke sigprocmask() for this particular
 >>>>>>> lock perhaps? I'm not sure how easy that would be to achieve given the
 >>>>>>> hooks to allow the thread library to overload the locking routines.
 >>>>>>> Also, this doesn't explain the lack of sigprocmask() calls under i386.
 >>>>>>> FreeBSD/i386 should be using the same locking code and thus invoking
 >>>>>>> sigprocmask() for each exception as well.
 >>>>>> Throwing an exception during asyncronous signal execution rises undefined
 >>>>>> behaviour, AFAIK. sigprocmask() is there to support libc_r, and cannot
 >>>>>> be removed as far as we need to provide FreeBSD 4.x compatibility.
 >>>>> Hmmm.  Why does libthr use sigprocmask() for its rtld locks then?  Is that 
 >>>>> just a copy-paste from libc_r that can be removed now?
 >>>> Hmmm^2. It seems it is there to prevent recursive entry into rtld from
 >>>> signal handler, that may reference yet unresolved symbol, e.g. libc
 >>>> syscall wrapper, from PLT. So my patch is wrong.
 >>> Presumably we could use a different type of lock that doesn't use
 >>> sigprocmask() to serialize calls do dl_iterate_phdr()? I'm not sure if
 >>> libthr would really need to overwrite the behavior of that lock or if
 >>> a simple atomic_cmpset()-based mutex would always be fine.
 >> During my porting of libunwind, I was told by libunwind maintainer
 >> that they have to call dl_iterate_phdr() from signal context to
 >> unwind, if libunwind is called from signal context.
 >>
 >> Apparently, glibc' dl_iterate_phdr() is not signal-safe, while our is.
 > 
 > [Revisiting this]
 > 
 > Do we know of any use cases where libunwind would be used from a signal
 > handler?  Could we instead simply declare it to be an unsafe API in a signal
 > context?  longjmp(3) isn't safe in a signal context and throwing exceptions
 > in a signal handler is undefined, so declaring libunwind to similarly be
 > unsafe may be fine.
 > 
 >>> OTOH, I'm not sure why libthr needs to use non-standard lock hooks at
 >>> this point as they don't seem to be markedly different from the ones
 >>> rtld uses.
 >> libthr locks provide exclusion both for other kernel-executed threads
 >> and signal handlers, while the rtld-default locks only protect against
 >> signal handlers and thus libc_r-style threads.
 > 
 > Oh, bah.  The rtld locks do use atomic operations that are thread safe,
 > but I missed that the 'oldsigmask' global needs to be per-thread.
 > 
 
 Current I am testing on a signal wrapper patch for libthr,
 as a side effect, the patch eliminates the need of sigprocmask for
 rtld lock.
 
 time costed by the example on my machine is:
  > time ./testexcept
 
 0.437u 0.000s 0:00.43 100.0%	5+5120k 0+0io 0pf+0w
 
 The problem still exists if the program does not create a second thread,
 because I have trouble to enable libthr's rtld lock in
 _libpthread_init() which has __attribute__ ((constructor)), this
 means rtld is in critical region, and I can not use run _thr_rtld_init() 
 to set rtld locks at that time, chicken-egg problem.
 
 The patch is mainly used for fixing thread cancellation race which
 is caused by signal, yes, signal is always a kind of pain for thread
 library.
 
 http://people.freebsd.org/~davidxu/patch/signal_wrapper.patch