Time to abandon recursive pulling of dependencies?

[[LoN]Kamikaze]

Kris Kennaway wrote:
> On Sun, May 13, 2007 at 10:44:19AM +0200, [LoN]Kamikaze wrote:
>> [LoN]Kamikaze wrote:
>>> Stephen Montgomery-Smith wrote:
>>>> OK chaps, this is what I came up with.  So for example, if I do "make
>>>> install" on /usr/ports/x11/xorg (having made all the dependencies), on
>>>> my computer it turns the pkg_create from taking about 4 minutes to the
>>>> blink of an eye.  Now people need to figure out how to speed up the
>>>> "make package-depends" in bsd.ports.mk, but that is beyond my abilities.
>>> I will give that a try, I've been experimenting with make a lot lately. Thank
>>> you for your patch, the second attempt seems to work fine on my system.
>> So far I've got a 20% performance increase, not really satisfying. The trouble
>> is that for every dependency the whole bsd.port.mk and whatever else is pulled
>> in by the dependency has to be parsed again. So the only way to make a
>> significant improvement is to have a short bsd.port.mk.
> 
> Again, what evidence do you have that *that* is really the problem?
> Please try to be more scientific in your approach, it will help all of
> us to figure this out.
> 
> Kris

Well, it takes about 1 minute and 15 seconds for x11/xorg on my system, which
has 273 dependencies, with my optimizations it still takes 60 seconds.

Parsing bsd.port.mk takes an average of 0.2 seconds on my system. 0.2 * 273 =
54.6 seconds. Which looks like the major part of computation time to me.

The only way to avoid this I see, is to abandon the recursive pulling in.
Packages wouldn't know all their dependencies any more, but the dependencies
would still know what they depend on and so forth.

I will just change my system and see weather it breaks. I suspect that some
things like pkg_add -r would have to be slightly adjusted to support such a change.