CFT: New version of portmaster in svn

[jhell]

On 07/23/2010 15:00, Doug Barton wrote:
> The problem is that in order to accomplish that portmaster would have to
> check every port in the tree. Assuming 22,000 ports, and that any given
> port is equally likely to fall anywhere in the tree, on average you'd
> have to search 11,000 ports for every distfile that is not related to a
> port installed on that system.
> 
> Now what I _could_ do is use the same technique I use in
> --clean-distfiles (create a text file with the distfile information) but
> instead of limiting it to installed ports, do all of them. I have never
> even tested that to see how long it would take, but I suppose I could
> take a look.

Hi Doug Barton, Ports@,

Here is a suggestion that I have just played around with involving
locate(1) with environment LOCATE_PATH set in portmaster & ports-side
locate.updatedb(8) with LOCATE_CONFIG before the port snapshots are made.

Server side: ( at port snapshot creation )
	LOCATE_CONFIG=/usr/ports/locate.rc /usr/libexec/locate.updatedb
		locate.rc:
			FCODES="/usr/ports/locate.database"
			SEARCHPATHS="/usr/ports"

real 149.79
user 3.51
sys 27.53
      2076  maximum resident set size
        37  average shared memory size
      3946  average unshared data size
       134  average unshared stack size
      2893  page reclaims
       335  page faults
         0  swaps
         0  block input operations
         0  block output operations
         0  messages sent
         0  messages received
         0  signals received
     35875  voluntary context switches
     38020  involuntary context switches


In portmaster(8):
	LOCATE_PATH=/usr/ports/locate.database

Generate a list of distinfo files and create a distfile index using
``SIZE''.

for file in $(locate "/usr/ports/*/*/distinfo"); do
	grep "SIZE " $file >>DISTINDEX.8
done

Use the output from that to judge if the file in /usr/ports/distfiles
needs to exist or of the file does not meet the SIZE requirements & you
can kill two birds with one stone.

You could always use the MD5 or SHA256 sum that's recorded but that may
be IMO a slight bit of overhead.

This may not be an exact proccess right now but it sure would be nice to
have a locate database in ports that could be updated by  (make
locatedb) or something similar.


Just for creating the list of distfiles from the SIZE contents:
$ du -sh DISTINDEX.8.1.gz
512B    DISTINDEX.8.1.gz

time -alhp ./createindex.sh
real 490.99
user 28.03
sys 117.14
      1820  maximum resident set size
       228  average shared memory size
      2297  average unshared data size
       133  average unshared stack size
   2353331  page reclaims
         0  page faults
         0  swaps
         0  block input operations
         0  block output operations
         0  messages sent
         0  messages received
         0  signals received
     77884  voluntary context switches
    165392  involuntary context switches


Regards,

-- 

 jhell,v