bin/122061: dlsym() is very slow when a symbol cannot be found

[Jacques Garrigue]

>Number:         122061
>Category:       bin
>Synopsis:       dlsym() is very slow when a symbol cannot be found
>Confidential:   no
>Severity:       non-critical
>Priority:       medium
>Responsible:    freebsd-bugs
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Tue Mar 25 00:50:04 UTC 2008
>Closed-Date:
>Last-Modified:
>Originator:     Jacques Garrigue
>Release:        7.0-RELEASE
>Organization:
Nagoya University
>Environment:
FreeBSD tet4.kurims.kyoto-u.ac.jp 7.0-RELEASE FreeBSD 7.0-RELEASE #0: Fri Feb 29 17:32:31 JST 2008     garrigue at tet4.kurims.kyoto-u.ac.jp:/usr/src/sys/i386/compile/TET  i386

>Description:
Calling dlsym() on a shared object that doesn't contain a symbol is very slow
compared to 6.2-RELEASE.

I discovered the problem when compiling the port x11-toolkits/ocaml-lablgtk2.
Compilation works normally at first, then stalls on the command:

ocamlmktop -I +lablGL -thread  -o lablgtktop unix.cma threads.cma lablgl.cma \
           -I . lablgtk.cma lablgtkgl.cma lablglade.cma lablgnomecanvas.cma lablgnomeui.cma lablpanel.cma lablrsvg.cma lablgtkspell.cma gtkThread.cmo

It succeeds eventually, but takes more than 1 minute rather than the few seconds
it took before. To understand what is happening, you have to know the internals
of the ocaml compiler: each of the libraries (.cma) here are requiring a shared
object each, and try to lookup symbols into them (about 2000 total). But rather
than looking the symbol where it is known to be, all the shared objects, and
then dlsym is used to manually look for it in each shared object individually.
So if you're out of luck you end up doing about 20000 failing lookups for 2000
successes.
Note that with a simple patch on ocaml, which reorders the list of shared
objects so that the last successful object is looked at first, the above
command line takes only 1 second.
>How-To-Repeat:
/* Compile and execute the following program */

#include <dlfcn.h>

int main()
{
     int i;
     void *handle = dlopen("/usr/local/lib/libgtk-x11-2.0.so", RTLD_NOW);

     for (i=0; i<20000; i++)
          dlsym(handle, "zorglub"); // takes 5 seconds
     // dlsym(handle, "gtk_main"); // takes 0.1 seconds

     return 0;
}


>Fix:


>Release-Note:
>Audit-Trail:
>Unformatted: