Time to abandon recursive pulling of dependencies?

[Stephen Montgomery-Smith]

Kris Kennaway wrote:
> On Sun, May 13, 2007 at 10:44:19AM +0200, [LoN]Kamikaze wrote:
>> [LoN]Kamikaze wrote:
>>> Stephen Montgomery-Smith wrote:
>>>> OK chaps, this is what I came up with.  So for example, if I do "make
>>>> install" on /usr/ports/x11/xorg (having made all the dependencies), on
>>>> my computer it turns the pkg_create from taking about 4 minutes to the
>>>> blink of an eye.  Now people need to figure out how to speed up the
>>>> "make package-depends" in bsd.ports.mk, but that is beyond my abilities.
>>> I will give that a try, I've been experimenting with make a lot lately. Thank
>>> you for your patch, the second attempt seems to work fine on my system.
>> So far I've got a 20% performance increase, not really satisfying. The trouble
>> is that for every dependency the whole bsd.port.mk and whatever else is pulled
>> in by the dependency has to be parsed again. So the only way to make a
>> significant improvement is to have a short bsd.port.mk.
> 
> Again, what evidence do you have that *that* is really the problem?
> Please try to be more scientific in your approach, it will help all of
> us to figure this out.
> 
> Kris

I looked inside the bsd.ports.mk, and put in a few ${ECHO_MSG} 
statements here and there.  I also put a few printf statements in 
pkg_create.  I found out that there are two parts that take a long time:

1.  Pulling in the dependencies.  This is in effect doing a "make 
package-depends".  You can do this for yourself, and see that it takes a 
long time.  I honestly don't see how to make this faster, as presumably 
it involves calling "make" on all the dependency ports, and I would 
guess that each new invocation of make adds considerable overhead just 
loading the program.

2.  Sorting the dependencies in pkg_create.  My fix now cuts this out 
completely.  Thus there is no need to change the structure of 
/var/db/pkg to use databases or anything like this.

Stephen