Create freebsd-testing-results@ list for Jenkins results?

[Craig Rodrigues]

Hi,

Yes, I am opposed to this.

The minute you push these email notifications to a separate list,
then the whole utility of Jenkins goes down, because most people won't
be subscribe to the list.

The whole point of running a CI system like Jenkins is to unfortunately be
a bit annoying
and "in your face".  FreeBSD doesn't have a QA department to triage
failures and notify
developers, so sending e-mails to
targeted lists is important.  As it is, FreeBSD people don't subscribe to
all the same lists, so no matter what,
you can't make everyone happy.

At this link:
https://www.freebsd.org/doc/en/books/handbook/current-stable.html

it mentions that bulletins about the state of the FreeBSD-CURRENT branch
go to the freebsd-current@ list.  So that's why it is OK for e-mail
notifications
from the old Tinderbox or Jenkins to go to that list.

I have received e-mails privately thanking me for setting this stuff up.
Some folks (who are not FreeBSD committers) monitor lists like
freebsd-current@,
and when they see Jenkins failure e-mails
on the list, that gives them a clue that they should hold off on updating
the tree.

So freebsd-current@ is used to notifiy committers *and* non-committers
about the
state of the branch.

jenkins is already configured to send e-mails to several lists, such as
freebsd-current, freebsd-stable, freebsd-doc, freebsd-i386, depending on
the build job,
so we don't send every e-mail to freebsd-current for all the builds that we
do.

So Jenkins is doing its job, even if it annoys some people and makes some
people
like Steve Kargl unhappy.

What we can consider doing is reducing the number of lines in the e-mall
notification,
to prevent very large e-mails from being sent to the list.  There may be
tunables
in Jenkins that we can look at for this.  That is worth exploring.

No matter what, you can't make everyone happy with build break e-mails.

However, pushing them off to the side is not the direction we should be
going in.

--
Craig


On Wed, May 20, 2015 at 10:46 PM, Garrett Cooper <yaneurabeya at gmail.com>
wrote:

> Hi all,
>         As some know who are subscribed to freebsd-current, the list gets
> spammed a lot with test results (pass/fail transitions mostly) for the
> Build_Image_and_Run_Tests_in_Bhyve_HEAD job. I was wondering if anyone
> would be opposed to creating another list to capture the results, e.g.
> freebsd-testing-results at FreeBSD.org, and if so, would you subscribe to
> the list?
> Thanks!
> -NGie
>