Looking for speed increases in "make index" and pkg_version for ports

[Mike Meyer]

In <465B4A84.6050407 at dlr.de>, Hartmut Brandt <hartmut.brandt at dlr.de> typed:
> Mike Meyer wrote:
> > In <465AB421.10802 at dlr.de>, Hartmut Brandt <hartmut.brandt at dlr.de> typed:
> >> 1. make and its sub-makes for a) reading the file; b) parsing the file
> >> (note that .if and .for processing is done while parsing); c) processing
> >> targets.
> > 
> > Make and submakes have been gone over already. See <URL:
> > http://miller.emu.id.au/pmiller/books/rmch/ >.
> > 
> > I'm not sure it can be applied to the ports tree, though. I haven't
> > looked into it, but recalled this paper when you mentioned measuring
> > makes and sub-makes.
> Unfortunately you deleted the sentence before, so I rephrase it: before 
> looking into optimizations find out where the time is actually spend - 
> how many seconds of the hours the process takes, are actually spent in 
> make and sub-makes. If the entire process takes 2 hours of which the 
> makes take 20 seconds then by enhancing performance of make by 50% you 
> win 10 seconds. This is probably not worth a single line of additional code.
> 
> The paper you point to talks about something entirely different.

It think we're talking about two different things. You're talking
about the efficiency of make, whereas he's talking about the
efficiency of make. Um, wait.

You're talking about what I'll call the *internal* efficiency of make,
defined as how fast it does the things it does. He's talking about
what I'll call the *external* efficiency of make, which is how well it
does at doing the minimum amount of work it needs to do. I hope you
can see where the confusion comes from.

In particular, he talks about how recursive makefiles screw up
evaluating complex variables, causing them to be executed multiple
times. So if you're running a makefile to pull some variables value,
as opposed to do real commands, and your entire process takes 2 hours
and the Makefile takes 20 seconds, but it evaluates all the variables
twice, then by fixing your makefile you win at least 59 minutes and 50
seconds. I think cutting the run time by 50% is worth some work.

Benchmarking can help you decide which things it pays to work on if
all you're worried about is the internal efficiency. However, the goal
is to make the process faster, so we need to worry about the external
efficiency as well. The problem here is that the worse it is, the less
it looks like you stand to gain by looking at your makefile when you
look at the benchmarks.

Given that the ports system has both highly complex variables and is
very recursive, I believe that it warrants investigation if you're
going to work on making make in the ports faster.

	<mike
-- 
Mike Meyer <mwm at mired.org>		http://www.mired.org/consulting.html
Independent Network/Unix/Perforce consultant, email for more information.