Building the INDEX

[Michel Talon]

Hello,

Some time ago i have reported about a python script able to build the INDEX.
Vasil Dimov pointed out that it had a performance problem, compared to
make index INDEX_JOBS=3

This is of course because the make index command, however ugly it is
programmed, achieves a lot of parallelism, which is difficult to achieve
with python.

Anyways, numbers floated around about the build time necessary to compute the
INDEX, and the problem this is causing to the ports team. People mentioned
times of the order of 30' for the present day ports tree with 16000 ports.

Today someone lent me a Core 2 Duo machine on which i have loaded FreeBSD-64.
Nice result is that both my python program (that i have multithreaded - but
there is room for improvement) and make index INDEX_JOBS=3 complete
building the INDEX in 8 minutes. Of course this program requires python, but
make index requires perl, which is not better. In view of this, one may
imagine that a properly threaded C program, which would allow better
parallelism, could build the INDEX in perhaps 6 minutes. Frankly, these
Core 2 Duo machines are absolutely fantastic. Basically my python program runs
more than 3 times faster than on a P4, 3 Ghz, hyperthreaded.





-- 

Michel TALON