[RFC] Un-staticise the toolchain

[Gary Palmer]

On Tue, May 01, 2012 at 01:53:58PM +0200, Erik Cederstrand wrote:
> Den 01/05/2012 kl. 07.52 skrev Tim Kientzle:
> > 
> > On Apr 30, 2012, at 6:41 AM, Erik Cederstrand wrote:
> >> 
> >> Can anyone explain to me why the dynamically linked version is significantly slower? What are the extra steps involved compared to a statically linked binary?
> > 
> > At the risk of dramatically over-simplifying?.
> > 
> > When a static binary is started by the kernel, it does the following:
> >  * Initializes some libc internals.
> >  * Calls main.
> > 
> > When a dynamic binary is started by the kernel, it does the following:
> >  * Initializes some libc internals.
> >  * For every dynamic library referenced by this executable:
> >       - loads the dynamic library into memory
> >       - fixes up references
> >  * Calls main
> > 
> > The process of loading the required libraries and fixing up references
> > can be quite time-consuming.
> 
> Thanks for the explanation. In the previous 'make index' benchmark by
> Chris, make is called very often, which means the dynamic libraries
> should already be loaded into memory after the first run, right? Which
> means the extra time is being spent fixing up references?

To an extent, yes.  However the kernel still has to do some work to resolve
the filename for the library to an inode, check to see if that inode is
already loaded into memory, and then map the pages for the shared libraries
into the new process that is requesting them.  I may be misremembering (it
is many many years since I had anything to do with the VM system) but I
think some of that effort is non-trivial.

If you want a high-level view of what goes on run

ldd `which ls`

check that it has libraries to load and doesn't say "not a dynamic ELF
executable", and then run: 

ktrace ls
kdump | more

All the system calls related to resolving and loading shared libraries
take time.  I realise "ls" is not "make", but it should give you an
idea.

Gary