repeatedly opening the same .so(s) is slow?

[Peter Jeremy]

On Mon, 2006-Mar-20 10:49:57 -0600, Eric Schuele wrote:
>I have a port (gnucash) which takes 3-4 minutes to open on a 2.6GHz 
>machine.  It used to take 15-20 seconds till all of the libtool changes.

It takes 15 minutes on a -current Athlon XP-1800 and about 2 minutes on
a 2.2GHz AMD64 running -stable.

>I have no idea if the symptom is related to libtool or not.

I initially thought it was libtool related but now I'm uncertain.  I
didn't just upgrade libtool, I upgraded quite a few other ports at the
same time.  On the not-libtool side, ade@ says that he hasn't seen this
behaviour with other libtool/libltdl ports and I've found that guile
include it's own libltdl code (based on libtool).  I'm not sure if this
is gnucash specific or affects other guile applications.

>Using truss, I can see that gnucash/guile is trying to open a dozen or 
>two files, repeatedly.  It fails attempting to open it the first few 
>times everytime it tries to access it, because it is traversing the 
>LD_LIBRARY_PATH:

Worse than that, it's expanding LD_LIBRARY_PATH using additional
paths embedded in the .la files that it's opening.

>Now I said a dozen or two files repeatedly.  It is 12-20 files maybe... 
>but it is attempting to open them *hundreds of thousands of times*!  It 
>goes on and on and on...

I took a complete ktrace of the startup and there are 24e6 NAMI events
with the top files tested 2,000,000 times.

>  I have 
>thought of placing symlinks in the folder(s) where it first looks for 
>any given file, to make sure it finds the file... but this does not seem 
>quite right either.

It's definitely a hack.  I tried something like this and it didn't
help much.  The code still wants to open libraries multiple times.

I've been looking at adding caching to lt_dlopenext() and my first
attempt went much faster but blew up because I wasn't correctly
handling open/close/open sequences (libm is opened and then closed
42,000 times).  I think this is the way forward but need to find
the time to understand ltdl.[ch] (~4800 lines).

>What I'm wondering is.... what is the lists opinion on how to best fix 
>this type of a problem.  Is this even the cause of my long startup?

Any system calls involving opening pathnames are expensive, even with
the namei cache.  Having 4 orders of magnitude too many is a destinct
problem.

>I have spoken with one or two of the gnucash devs, they seem to think 
>this is unique to FreeBSD, meaning they have not seen this problem on 
>any other platform.  They said it might have to do with how FreeBSD 
>handles how files are opened up many times recursively!?

Possibly Linux can more efficiently handle opening a non-existent file
but the underlying problem is that there are far too many system calls
being executed during the gnucash startup.  It would be interesting to
get a truss of gnuash starting on another OS (unfortunately, I don't
have access to any Linux systems) and/or some other guile applications.

>If there is a more appropriate list, please let me know.

-ports may be better.

-- 
Peter Jeremy