Differences between 'make index' and 'portsdb -U'

[Kris Kennaway]

I've been looking at index builds a fair bit recently, and I've
noticed a number of problems with both 'make index' and 'portsdb -U'
(part of portupgrade).

1) 'make index' doesn't quote whitespace in COMMENT entries, but
'portsdb -U' does.  There is probably some missing quoting in the perl
script in the 'describe' target of bsd.port.mk, or the make_index
script.

The first line is from portsdb, the second from 'make index'; notice
the additional space after the period in the comment.

-lrzsz-0.12.20_1|/usr/ports/comms/lrzsz|/usr/local|Receive/Send files via X/Y/ZMODEM protocol.  (unrestrictive)|/usr/ports/comms/lrzsz/pkg-descr|dinoex at FreeBSD.org|comms|||http://www.ohse.de/uwe/software/lrzsz.html
+lrzsz-0.12.20_1|/usr/ports/comms/lrzsz|/usr/local|Receive/Send files via X/Y/ZMODEM protocol. (unrestrictive)|/usr/ports/comms/lrzsz/pkg-descr|dinoex at FreeBSD.org|comms|||http://www.ohse.de/uwe/software/lrzsz.html
 
2) portsdb -U tries to build the index using 'make -j3'.  This causes
the output from the child make processes to sometimes overlap,
generating corrupted INDEX entries and errors (I tried a similar
optimization for 'make index', but encountered the same problem.  I
think it might be related to printing lines over a certain length).

3) portsdb -U doesn't record PATCH_DEPENDS entries, 'make index' does.

4) portsdb -U doesn't accept (valid) dependency lines like the
following (from p5-XML-Xerces):

BUILD_DEPENDS=  ${NONEXISTENT}:${PORTSDIR}/textproc/xerces-c2:"configure TRANSCODER=native"

This gives a spurious error during the build:

p5-XML-Xerces-2.3.0.4_2:"" non-existent -- dependency list incomplete

5) portsdb -U is now about 10% slower than 'make index', because it
uses a custom makefile to recurse through the tree, and doesn't make
use of the optimizations recently added to bsd.port.mk.  The portsdb
method may once have been an optimization relative to b.p.m, but it is
not any longer.

citusc17# /usr/bin/time make index
Generating INDEX-5 - please wait.. Done.
     3282.09 real      2334.77 user       658.47 sys
citusc17# /usr/bin/time portsdb -U
Updating the ports index ... p5-XML-Xerces-2.3.0.4_2:"" non-existent -- dependency list incomplete
done
     3591.99 real      2269.40 user      1075.79 sys

Modulo the cosmetic bug #1 above, this all counts in favour of using
'make index' to build your indexes, and against using 'portsdb -U'
until these problems are resolved.  It would probably be best to just
change portsdb to call 'make index' internally for the index builds.

Kris
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 187 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-ports/attachments/20040207/ddef6959/attachment-0001.bin