Time to abandon recursive pulling of dependencies?

[Stephen Montgomery-Smith]

Stephen Montgomery-Smith wrote:
> [LoN]Kamikaze wrote:
>> Stephen Montgomery-Smith wrote:
>>> Basically I think we are stuck on making "make package-depends" go any
>>> faster.
>>>
>>> However I do think that the modifications I made to pkg_create go a 
>>> very
>>> significant way to solving the problem of registration taking so 
>>> very long.
>>>
>>> Stephen
>>>
>>
>> You are right about invoking make being the problem. However, I was 
>> able to
>> increase the performance of package-depends by 20 to 30 percent, by 
>> avoiding
>> recursive make calls and moving the recursion into the stack of the 
>> shell
>> invocation.
>>
>> With my changes there's less than 10% overhead over the sum of make 
>> calls. I.e.
>>  for 56 seconds of make calls (x11/xorg on my system) there are only 
>> 4 more
>> seconds of processing, instead of 15 to 20 seconds.
>>
>> The original make package-depends does some things I don't 
>> understand. Once
>> I've figured them out I will post my little patch here.
>>
>>
>
>
> Someone pointed out that what I was proposing in +DEPENDENCIES is 
> already to be found in +CONTENTS.  So here is a proof of concept patch 
> to /usr/ports/Mk/bsd.port.mk (proof of concept because no error 
> checking, and things like that).
>
> For me it makes registration about 3 times faster.

And also, the only reason it goes slow is because it has to do
(cd $$dir; make -V PKGNAME)
for every dir in _LIB_RUN_DEPENDS.  But if instead we kept a file in 
/var/db/pkg called something like +PACKAGE_NAMES, where as each port is 
created we add in a one line entry with this mapping of origin to 
package name, then registration would take less than a second.  It also 
has the advantage that the created +REQUIRED_BY really would be an 
accurate reflection of the true state of affairs rather than what it 
should be (meaning that if you cvsup ports and then build on top of old 
dependencies, +REQUIRED_BY would still be accurate).

Stephen