About ports QA by 'port test'

[Garrett Cooper]

2010/3/8  <chukharev at mail.ru>:
> Recently I've been trying to run some additional tests during updating
> the ports I have installed on my computer. That means I select a number
> of oldest installed ports (using dates of directories in /var/db/pkg/),
> and for each of them run 'port test' (ports-mgmt/porttools) and then
> 'portupgrade -f'.
>
> FYI, the 'port test' uses portlint, builds in a different place, installs
> into a different place, packages, then de-installs and checks for left
> files. No jail, no chroot.
>
>  From what I have seen till now, about 25% of the ports do not pass the
> test, either for fatal errors from portlint or due to errors exhibited
> because of PREFIX and PKG_DBDIR variables. And this is done only for
> the ports I have successfully installed on my system for some reason
> not connected to the QA purposes.
>
> I do not want to trouble the port maintainers with direct e-mails.
>
> I've been posting the results (and the original description I'm modifying
> now) to freebsd-qa@ for a while, but that turned out to be of some burden
> and also I received there no feedback at all.
> Therefore I set a webpage at my day-job where I will copy the results.
> I hope I will not need to keep the site for long, if running 'port test'
> is found useful, it should be added into QAT or tinderbox, I think. Though
> without jail it is faster I guess.
>
> The URL: http://kemia.me.tut.fi/~chu/FreeBSD.port_tests/
> Current statistics:

If the host was reachable...

> Ports having fatal errors:             109 (21 %)
> Ports having warnings:                 216 (43 %)
> Ports without fatal errors:            391 (78 %)
> Ports having warnings or fatal errors: 325 (65 %)
> Totally tested ports:       500
>
> Additionally to the above stats, java/jdk16 is interactive and hangs on
> input
> despite of BATCH=yes in /etc/make.conf.

Thanks,
-Garrett