(In)Stability of the Quarterly Branch

[Matthew Seaman]

On 2016/12/15 16:01, Olivier Duchateau wrote:
>> The problem is that there are no tests in FreeBSD ports. All source
>> based systems I've tested: pkgsrc, FreeBSD ports, OpenBSD, Gentoo;
>> FreeBSD is the one that have the most instability. Not to mention
>> committers that commit without testing the port, just look at
>> www/redmine to get your point of view on that issue.

> Are your serious when you said, there're no tests on FreeBSD ports. I
> can tell you Xfce ports are tested with FreeBSD i386 9.3 and amd64
> 11.0 machines (on real hardware, no virtualization), and on poudriere
> with Gtk+ 3.20 (port version is not not in ports tree, it's defaut
> toolkits for the next stable release 4.14).
> 
> For the LXQt desktop is the same thing (tested with official ports
> tree Qt5 and which one in plasma5 branch (on KDE repository).
> 
> I'm also working on the Pantheon desktop (desktop environment of
> Elementary OS, I use Vala 0.30.2 and Vala 0.34.4, in order to test
> stability of applications.
> 
> I use also OpenBSD macppc, it's piece of shit. WebKit browers are
> broken, Xfce components crash often, stable branch is outdated, fix
> are not propagated in stable branch. Personally I prefer the FreeBSD
> scheme, because I'm sure it's quite stable.

Most port committers will run compile tests any time they update a port:
the better ones will test compilation on all supported FreeBSD versions
and all hardware architectures they have access to (ie. generally i386
and amd64).

Additionally the package build cluster will rebuild any modified ports
within a few days for all of the OS versions and architectures the
project tries to provide ports for: that's yet another level of
validating the coding of the port itself.

However, I believe the OP's point is that *we do not routinely run the
software's own built-in regression tests for the packages we succeed in
building*.  This is something that is slowly coming.  For instance, you
can run 'make test' for many python, ruby or perl packages and see those
tests being run.  TEST_DEPENDS is pretty much standardized as the way to
install dependencies required for testing nowadays.

Yet another layer of package validation would be very good to have, but
it isn't routine yet.

	Cheers,

	Matthew


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 972 bytes
Desc: OpenPGP digital signature
URL: <http://lists.freebsd.org/pipermail/freebsd-ports/attachments/20161215/ca0057ed/attachment.sig>