Call for Help: openjdk8 tests under Continuous Integration

[Craig Rodrigues]

Hi,

Thanks, Brian!

Well, first of all, we need someone to figure out how to run the jtreg
tests under FreeBSD,
and make that process reproducible.  Would you be willing to do the initial
investigation for this?
If you look at Kip Macy's postings on the freebsd-java list in October, you
can see that he
managed to get the tests to run, so hopefully Kip can answer any questions
about what he did to get
the tests to run.

I'm not familiar with jtreg.  What ports are you thinking that we need to
build with poudriere in order to run it?

Le me try to answer your questions:

1)  Let's test on amd64 first, just to iron out all the issues.  Once we
get that working, we can add more platforms, such as i386.
2)  For an initial effort, maybe we can schedule the tests to run once a
week, via the Jenkins job scheduler.  We can see if
     it makes sense to run the tests more or less frequently after that.
3)  I don't know what format the jtreg test output produces.  If it can
produce JUnit XML output, that would be ideal because
     it can be imported directly into Jenkins.  If it does not do that,
then even if we make the text output of the test logs available online,
     that is a good start.

--
Craig

On Sun, Nov 23, 2014 at 9:15 PM, Brian Gardner <openjdk at getsnappy.com>
wrote:

> Hello,
> I'd like to help.  Sounds like those scripts would be fairly easy to
> write.  Would it be a good idea to try and decouple building of the ports
> (using poudriere) from the actual running of the jtreg tests.  In this plan
> you'd have a separate job that builds and maintains an up-to-date
> repository of  packages for the supported platforms (10.0 amd64, 9.3 i386),
> then as part of running tests you'd run pkg to update the packages as the
> first step.
>
> I have some questions:
> 1) What platforms will be tested?
> 2) How often will tests run / What will trigger test runs?
> 3) How will results of the jtreg tests be presented on Jenkins?
>
> Brian Gardner
>
>
> In Response To:
>
> Hi,
>
> Recently Kip Macy tried to get the openjdk8 tests working
> under FreeBSD.  He struggled a bit:
> https://lists.freebsd.org/pipermail/freebsd-java/2014-October/010928.htmlhttps://lists.freebsd.org/pipermail/freebsd-java/2014-October/010929.htmlhttps://lists.freebsd.org/pipermail/freebsd-java/2014-October/010930.html
>
> I think it would be good to get these tests running under FreeBSD
> continous integration.  These tests stress things like threading, memory
> management, networking, and virtual memory.  Running these tests
> would go a long way to improving Java support under FreeBSD,
> because it would give a better idea of what can be fixed in FreeBSD itself.
>
> I am a bit overstretched at the moment to take this on.
>
>
> I need scripts that can do the following:
>
> (1)  Download the necessary packages, openjdk8, jtreg, etc.
> (2)  Apply necessary patches to packages in (1)
> (3)  Build the packages in (1) after patches applied
> (4)  Run the tests
>
> I would also like:
> (A)  Scripts should be committed to https://github.com/freebsd/freebsd-ci
> (B)  Script should run under FreeBSD
> (C)  Scripts should be easy to run under a Jenkins job at
>        https://jenkins.freebsd.org
> (D)  Followup status and discussion should happen on
>        the freebsd-testing at freebsd.org <https://lists.freebsd.org/mailman/listinfo/freebsd-testing> mailing list
>
> Is anyone interested in helping out with this?  Starting with some
> of the things that Kip did would be a good place to start.
>
> Thanks.
> --
> Craig
>
>
>
>
> On Nov 23, 2014, at 8:49 PM, Craig Rodrigues <rodrigc at FreeBSD.org> wrote:
>
> Hi,
>
> Read this:
>
> https://lists.freebsd.org/pipermail/freebsd-testing/2014-November/000668.html
>
> and reply to that posting on freebsd-testing at freebsd.org (subscribe to
> that list if you
> are not on it) with the items that you think that you can
> complete.
>
> --
> Craig
>
> On Sun, Nov 23, 2014 at 8:28 PM, Brian Gardner <openjdk at getsnappy.com>
> wrote:
>
>> Hi Craig,
>> I would like to help.  Let me know what I can do.
>>
>> Brian Gardner
>>
>> > On Nov 22, 2014, at 4:40 PM, Craig Rodrigues <rodrigc at FreeBSD.org>
>> wrote:
>> >
>> > FYI,
>> >
>> >
>> https://lists.freebsd.org/pipermail/freebsd-testing/2014-November/000668.html
>> >
>> > Please send followups to freebsd-testing at freebsd.org.
>> >
>> > --
>> > Craig
>> > _______________________________________________
>> > freebsd-java at freebsd.org mailing list
>> > http://lists.freebsd.org/mailman/listinfo/freebsd-java
>> > To unsubscribe, send any mail to "freebsd-java-unsubscribe at freebsd.org"
>>
>>
>
>