Looking for speed increases in "make index" and pkg_version for ports

[Stephen Montgomery-Smith]

I'm looking for something that will work with the existing framework. 
But yes, I get the feeling that maybe using "make" to process the ports 
might be the source of the problem.  Make is a program primarily 
designed for figuring out which was made first, the target or the 
source, but in the ports what we really want is a scripting language 
that presides over "cd WKSRC; make".

(P.S. sorry for top-posting, but I am following your lead.)


Bakul Shah wrote:
> Not quite what you asked for but...
>  
> Given the size and complexity of the port system I have long
> felt that rather than do everything via more and more complex
> Mk/*.mk what is is needed is a ports server and a thin CLI
> frontend to it.
> 
> This server can store dependency data in an efficient manner,
> deal with conditional dependencies, port renames, security
> and what not.  It can build or fetch or serve packages,
> handle updates etc.  Things mentioned in UPDATING file can
> instead be done by the server.  In general it can automate a
> lot of stuff, remove error prone redundancies etc.  If it is
> small enough and written in C, it can even be shipped with
> the base system instead of various pkg_* programs.
> 
> It can provide two interfaces, one for normal users (with
> commands like add, check, config, delete, info, search,
> update, which) and one for port developers (command for
> adding/remove/renaming ports, etc.).  Initially it must work
> with existing Makefiles.
> 
>> I have been thinking a lot about looking for speed increases for "make 
>> index" and pkg_version and things like that.  So for example, in 
>> pkg_version, it calls "make -V PKGNAME" for every installed package. 
>> Now "make -V PKGNAME" should be a speedy operation, but the make has to 
>> load in and analyze bsd.port.mk, a quite complicated file with about 
>> 200,000 characters in it, when all it is needing to do is to figure out 
>> the value of the variable PKGNAME.
>>
>> I suggest rewriting "make" so that variables are only evaluated on a 
>> "need to know" basis.  So, for example, if all we need to know is 
>> PKGNAME, there is no need to evaluate, for example, _RUN_LIB_DEPENDS, 
>> unless the writer of that particular port has done something like having 
>> PORTNAME depend on the value of _RUN_LIB_DEPENDS.  So "make" should 
>> analyze all the code it is given, and only figure it out if it is needed 
>> to do so.  This would include, for example, figuring out .for and .if 
>> directives on a need to know basis as well.
>>
>> I have only poked around a little inside the source for make, but I have 
>> a sense that this would be a major undertaking.  I certainly have not 
>> thought through what it entails in more than a cursory manner.  However 
>> I am quite excited about the possibility of doing this, albeit I may 
>> well put off the whole thing for a year or two or even forever depending 
>> upon other priorities in my life.
>>
>> However, in the mean time I want to throw this idea out there to get 
>> some feedback, either of the form of "this won't work," or of the form 
>> "I will do it," or "I have tried to do this."
>>
>> Best regards, Stephen
>> _______________________________________________
>> freebsd-hackers at freebsd.org mailing list
>> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
>> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe at freebsd.org"
> 
>