Data corruption with 'portsdb -U'

[Kris Kennaway]

On Tue, Jan 27, 2004 at 03:43:05PM -0800, Kris Kennaway wrote:

> > That hasn't always been true. Make index would find 1 or 2 more ports 
> > than -U would. There are also times when make index falls flat and -U 
> > would still produce a useful INDEX. In addition, portsdb -U used to run 
> > quite a bit faster than make index did.
> 
> I'm seeing what look like data corruption from portsdb -U:
> 
> Updating the ports index ... p5-DateTime-Format-HTTP-0.35:"" non-existent -- dependency list incomplete
> ind/Rule.pm:/usr/ports/devel/p5-File-Find-Rule /foobar/lib/perl5/site_perl/5.6.1/File/Spec.pm:/usr/ports/devel/p5-File-Spec /foobar/bin/perl5.6.1:/usr/ports/lang/perl5: malformed entry: ind/Rule.pm:/usr/ports/devel/p5-File-Find-Rule /foobar/lib/perl5/site_perl/5.6.1/File/Spec.pm:/usr/ports/devel/p5-File-Spec /foobar/bin/perl5.6.1:/usr/ports/lang/perl5|
> 
> This may be because make_describe_pass1 is using 'make -j3' to try and
> optimize the build, and when I tried a similar trick with 'make index'
> on the weekend I also got this kind of corruption.  I think it's
> happening because the child makes are occasionally overlapping their
> I/O, so the parent ends up with a corrupted list where two
> neighbouring entries are intertwined.

Removing '-j3' from /usr/local/sbin/make_describe_pass1 allows the
INDEX build to finish without data corruption (and associated
'errors').  I don't know why portsdb uses -j, because the problems are
easily reproduced in my testing (on both 5.x and 4.x).

Kris
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 187 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-ports/attachments/20040128/90b8f3f8/attachment-0001.bin