Extra fields in INDEX

[Chris Rees]

On 1 September 2011 17:12, Eygene Ryabinkin <rea at freebsd.org> wrote:
> Chris, good day.
>
> Fri, Aug 19, 2011 at 03:32:43PM +0100, Chris Rees wrote:
>> I'm having a look at modifying INDEX to have a field at the end (to
>> cause minimum breakage), but I've discovered that in a few ports there
>> (6630 out of 22731) appears to be extra 'stuff' in fields past field
>> 10 (which is for the first WWW: line in pkg-descr).
>>
>> Can anyone with knowledge of make index explain why this is happening?
>> Using the command line at [1] gets a list
>>
>> [crees at hydra]~% alias findextrasinindex "< `make -C /usr/ports -V
>> INDEXDIR`/`make -C /usr/ports -V INDEXFILE` grep -n
>> '^\([^|]*|\)\{11\}[^|]\+'"
>
> This regexp won't catch empty field #12, so you're effectively counting
> the number of entries with non-empty field #12:
> {{{
> $ cat INDEX-9 | cut -f 12 -d '|' | grep -v '^$' | wc -l
>    6221
> $ grep -n '^\([^|]*|\)\{11\}[^|]\+' INDEX-9 | wc -l
>    6221
> }}}
>
> Every line in INDEX has 13 fields:
> {{{
> $ grep -n '^\([^|]*|\)\{12\}[^|]*$' INDEX-9 | wc -l
>   22287
> $ wc -l INDEX-9
>   22287 INDEX-9
> }}}
> as every line in 'make describe':
> {{{
> $ make -C archivers/fastjar describe | tr '|' '\n' | wc -l
>      13
> }}}
>
>> They have extra fields on the end ... Where have they come from??
>
> There are no extra fields, but /usr/ports/Tools/make_index just
> reorders them: the order becomes 0...6, 10, 11, 7...9, 12 if compared
> to the initial field indices.  What is effectively done by 'make
> index' in the case of fastjar index line is the following:
> {{{
> (make -C archivers/fastjar describe; \
>  make -C lang/perl5.8 describe; \
>  make -C devel/gmake describe) | \
> perl /usr/ports/Tools/make_index 2>/dev/null | \
> sed -e 's/  */ /g' -e 's/|  */|/g' -e 's/  *|/|/g' -e 's./.^A.g' | \
> sort -t '|' +1 -2 | sed -e 's.^A./.g' | grep ^fastjar
> }}}
> and '^A' here is the real '^A', not two chars '^' and 'A'.
>
> make_index respects the trailing fields (those that come after 13th)
> and they will be put to the end, so if you will add something
> as additional trailing field produced by 'make describe', it will
> survive piping through make_index and will be in its place in the
> INDEX file.

Thanks very much Eygene -- I'd tracked down all these by talking to
linimon, and neglected to reply to this email. I'm sorry I've wasted
your time :(

Also, you're right about the regexp, damnit!

Chris

-- 
Chris Rees          | FreeBSD Developer
crees at FreeBSD.org   | http://people.freebsd.org/~crees