FCP 20190401-ci_policy: CI policy

[Marcelo Araujo]

Em qui, 29 de ago de 2019 às 22:44, Marcelo Araujo <araujobsdport at gmail.com>
escreveu:

>
>
> Em qui, 29 de ago de 2019 às 20:03, Kristof Provost <kp at freebsd.org>
> escreveu:
>
>> On 29 Aug 2019, at 13:40, Konstantin Belousov wrote:
>> > On Wed, Aug 28, 2019 at 12:29:58PM +0800, Li-Wen Hsu wrote:
>> >> It seems I was doing wrong that just changed the content of this FCP
>> >> to "feedback", but did not send to the right mailing lists.
>> >>
>> >> So I would like to make an announcement that the FCP
>> >> 20190401-ci_policy "CI policy":
>> >>
>> >> https://github.com/freebsd/fcp/blob/master/fcp-20190401-ci_policy.md
>> >>
>> >> is officially in "feedback" state to hopefully receive more comments
>> >> and suggestions, then we can move on for the next FCP state.
>> >
>> > What problem does the document tries to solve ?  Or rather, do we
>> > really
>> > have the problem that it claims to solve ?
>> >
>> There are, somewhat regularly, commits which break functionality, or at
>> the very least tests.
>> The main objective of this policy proposal is to try to improve overall
>> code quality by encouraging and empowering all committers to investigate
>> and fix test failures.
>>
>
> Sure, but it doesn't sounds like you are empowering people to works in
> their spare time for the project, quite the opposite.
> Could you show something more feasible than "somewhat" regularly breaks
> functionality? Most of the time personally I'm running HEAD, built daily
> and I can't see this.
>
>
>>
>> >> From my experience, normal peer pressure is enough to get things
>> >> fixed
>> > quickly when it is possible to fix them quickly. If there is something
>> > more non-trivial, esp. in the tests and not the build, I am sure that
>> > a rule allowing anybody to do blind revert is much more harmful than
>> > having a test broken.
>> >
>> > More, I know that tests are of very low quality, which means that
>> > brokeness of the tests is not an indicator of anything until root
>> > cause
>> > is identified.
>> >
>> I’m not sure I agree with the characterisation that the tests are of
>> low quality. My own experience with the pf tests is that they test a
>> large section of the network stack and firewall code. They’ve
>> identified several very really issues (both pre- and post commit on the
>> epoch-isatin of the network stack, for example, as well as a fairly
>> important issue with IPv6 reassembly).
>> It’s certainly true that the pf tests often reveal issues that are not
>> in pf but in other code. I wouldn’t agree that this is a sign of low
>> quality tests, but instead I consider it a sign that we don’t have
>> enough tests for the network stack itself.
>>
>
> All the tests and CI are pretty new, or at least it is something new for
> most of everybody and people are getting used to that.
>
> I stop here, I would elaborate more, but after Ian's email, I think I
> don't need anymore.
>

One last thing I'd like to say: We should stop to try to control FreeBSD
contributions, it is not doing good for the project. Let it be, let it go,
there is no owner, just go and fix things.


>
>
>>
>> > Can we rely on the common sense of developers until there is indeed
>> > the
>> > visible problem ?
>> >
>> I don’t want to suggest that people simply don’t care about test
>> failures, because that’s clearly not true.
>>
>> On the other hand, I do think we can do better. There are at least two
>> open problem in the network stack that I currently can’t get anyone to
>> look at, and where I personally do not have sufficient context (or time)
>> to fix them myself. (#239380, #238870).
>>
>> This proposal isn’t a silver bullet, I don’t think there is such a
>> thing, but I do believe that elevating the visibility and importance of
>> test failures can help us improve overall quality.
>>
>> Best regards,
>> Kristof
>> _______________________________________________
>> freebsd-hackers at freebsd.org mailing list
>> https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
>> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe at freebsd.org
>> "
>>
>
>
> --
>
> --
> Marcelo Araujo            (__)araujo at FreeBSD.org     \\\'',)http://www.FreeBSD.org <http://www.freebsd.org/>   \/  \ ^
> Power To Server.         .\. /_)
>
>

-- 

-- 
Marcelo Araujo            (__)araujo at FreeBSD.org
\\\'',)http://www.FreeBSD.org <http://www.freebsd.org/>   \/  \ ^
Power To Server.         .\. /_)