Yet another INDEX builder.

[Kris Kennaway]

On Sat, Sep 30, 2006 at 12:25:19AM +0200, Thierry Thomas wrote:
> Le Ven 29 sep 06 ? 19:21:22 +0200, Michel Talon <talon at lpthe.jussieu.fr>
>  ?crivait?:
> > Hello,
> 
> Hi,
> 
> > i have written yet another index builder in python, which can be found here:
> > http://www.lpthe.jussieu.fr/~talon/build_index.py
> > I think it works relatively well and fast, and can provide building blocks for
> > doing a package updater alternative to portupgrade.  Of course, like the
> > documentation, it is completely free. I would be happy if some people could
> > test it on a real biproc machine, or a machine with fast disks, to see the
> > timings (*) one gets. On my machine, a P4 3Ghz, oldish, it takes 23 minutes to
> > complete building the INDEX, but the python script doesn't run more than 1
> > minute. Everything else is IO, i think. So one cannot expect any performance
> 
> It took longer on my P4 3.6GHz:
> 
> The tree (15689ports) takes  2789.84287214 seconds to explore.
> Now computing the recursively extended dependencies.
> Took  3.02815794945 seconds.
> Now converting to packages and sorting.
> Last phase takes  4.7659368515 seconds.
> Total time spent:  2800.94901609 seconds.
> python build_index.py  3024,42s user 2050,37s system 181% cpu 46:41,64 total
> 
> Remarks:
> - this is not a benchmark, and I was using the machine in the same time;
> - I have several local ports or local tweaks which have been reported as
> "The port             xxx/yyy is obsolete".

As Michel mentions, INDEX build is almost completely I/O bound, so it
almost entirely depends on the speed of your disk and how congested it
is from other I/O.  Thus it's not really meaningful to compare numbers
from one machine to another, but it still may be useful to compare to
'make index' times.  You should not expect a significant difference
unless something is going wrong though.

FYI, I'm not sure if the python version is parallelizable, but you do
get a small benefit from using parallelized 'make index' builds (via
INDEX_JOBS) on a typical SMP machine.

Kris
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 187 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-ports/attachments/20060929/38002d08/attachment.pgp