Call for Help: openjdk8 tests under Continuous Integration

Tue Nov 25 20:23:18 UTC 2014

Hi Craig,
I’ll continue the investigation, I’ve been following the openjdk8 posts from Kit.  I initially ported openjdk8 and was running jtreg tests as part of the process.  If I recall correctly the output was HTML but I think this is configurable, worse case I probably can convert it into supported xml format.  I’m thinking we can bake running tests into the openjdk8 port Makefile, and use poudriere testport and pass in a script that will upload the results into Jenkins.  That should make it really easy to run and maintain this on all supported platforms.  

Brian

> On Nov 25, 2014, at 12:11 PM, Craig Rodrigues <rodrigc at FreeBSD.org> wrote:
> 
> Hi,
> 
> Thanks, Brian!
> 
> Well, first of all, we need someone to figure out how to run the jtreg tests under FreeBSD,
> and make that process reproducible.  Would you be willing to do the initial investigation for this?
> If you look at Kip Macy's postings on the freebsd-java list in October, you can see that he
> managed to get the tests to run, so hopefully Kip can answer any questions about what he did to get
> the tests to run.
> 
> I'm not familiar with jtreg.  What ports are you thinking that we need to build with poudriere in order to run it?
> 
> Le me try to answer your questions:
> 
> 1)  Let's test on amd64 first, just to iron out all the issues.  Once we get that working, we can add more platforms, such as i386.
> 2)  For an initial effort, maybe we can schedule the tests to run once a week, via the Jenkins job scheduler.  We can see if
>      it makes sense to run the tests more or less frequently after that.
> 3)  I don't know what format the jtreg test output produces.  If it can produce JUnit XML output, that would be ideal because
>      it can be imported directly into Jenkins.  If it does not do that, then even if we make the text output of the test logs available online,
>      that is a good start.
> 
> --
> Craig
> 
> On Sun, Nov 23, 2014 at 9:15 PM, Brian Gardner <openjdk at getsnappy.com <mailto:openjdk at getsnappy.com>> wrote:
> Hello,
> I’d like to help.  Sounds like those scripts would be fairly easy to write.  Would it be a good idea to try and decouple building of the ports (using poudriere) from the actual running of the jtreg tests.  In this plan you’d have a separate job that builds and maintains an up-to-date repository of  packages for the supported platforms (10.0 amd64, 9.3 i386), then as part of running tests you’d run pkg to update the packages as the first step.
> 
> I have some questions:
> 1) What platforms will be tested?
> 2) How often will tests run / What will trigger test runs?
> 3) How will results of the jtreg tests be presented on Jenkins?
> 
> Brian Gardner
> 
> 
> In Response To:
> 
> Hi,
> 
> Recently Kip Macy tried to get the openjdk8 tests working
> under FreeBSD.  He struggled a bit:
> 
> https://lists.freebsd.org/pipermail/freebsd-java/2014-October/010928.html <https://lists.freebsd.org/pipermail/freebsd-java/2014-October/010928.html>
> https://lists.freebsd.org/pipermail/freebsd-java/2014-October/010929.html <https://lists.freebsd.org/pipermail/freebsd-java/2014-October/010929.html>
> https://lists.freebsd.org/pipermail/freebsd-java/2014-October/010930.html <https://lists.freebsd.org/pipermail/freebsd-java/2014-October/010930.html>
> 
> I think it would be good to get these tests running under FreeBSD
> continous integration.  These tests stress things like threading, memory
> management, networking, and virtual memory.  Running these tests
> would go a long way to improving Java support under FreeBSD,
> because it would give a better idea of what can be fixed in FreeBSD itself.
> 
> I am a bit overstretched at the moment to take this on.
> 
> 
> I need scripts that can do the following:
> 
> (1)  Download the necessary packages, openjdk8, jtreg, etc.
> (2)  Apply necessary patches to packages in (1)
> (3)  Build the packages in (1) after patches applied
> (4)  Run the tests
> 
> I would also like:
> (A)  Scripts should be committed to https://github.com/freebsd/freebsd-ci <https://github.com/freebsd/freebsd-ci>
> (B)  Script should run under FreeBSD
> (C)  Scripts should be easy to run under a Jenkins job at
>        https://jenkins.freebsd.org <https://jenkins.freebsd.org/>
> (D)  Followup status and discussion should happen on
>        the freebsd-testing at freebsd.org <https://lists.freebsd.org/mailman/listinfo/freebsd-testing> mailing list
> 
> Is anyone interested in helping out with this?  Starting with some
> of the things that Kip did would be a good place to start.
> 
> Thanks.
> --
> Craig
> 
> 
> 
>> On Nov 23, 2014, at 8:49 PM, Craig Rodrigues <rodrigc at FreeBSD.org <mailto:rodrigc at FreeBSD.org>> wrote:
>> 
>> Hi,
>> 
>> Read this:
>> https://lists.freebsd.org/pipermail/freebsd-testing/2014-November/000668.html <https://lists.freebsd.org/pipermail/freebsd-testing/2014-November/000668.html>
>> 
>> and reply to that posting on freebsd-testing at freebsd.org <mailto:freebsd-testing at freebsd.org> (subscribe to that list if you
>> are not on it) with the items that you think that you can
>> complete.
>> 
>> --
>> Craig
>> 
>> On Sun, Nov 23, 2014 at 8:28 PM, Brian Gardner <openjdk at getsnappy.com <mailto:openjdk at getsnappy.com>> wrote:
>> Hi Craig,
>> I would like to help.  Let me know what I can do.
>> 
>> Brian Gardner
>> 
>> > On Nov 22, 2014, at 4:40 PM, Craig Rodrigues <rodrigc at FreeBSD.org <mailto:rodrigc at FreeBSD.org>> wrote:
>> >
>> > FYI,
>> >
>> > https://lists.freebsd.org/pipermail/freebsd-testing/2014-November/000668.html <https://lists.freebsd.org/pipermail/freebsd-testing/2014-November/000668.html>
>> >
>> > Please send followups to freebsd-testing at freebsd.org <mailto:freebsd-testing at freebsd.org>.
>> >
>> > --
>> > Craig
>> > _______________________________________________
>> > freebsd-java at freebsd.org <mailto:freebsd-java at freebsd.org> mailing list
>> > http://lists.freebsd.org/mailman/listinfo/freebsd-java <http://lists.freebsd.org/mailman/listinfo/freebsd-java>
>> > To unsubscribe, send any mail to "freebsd-java-unsubscribe at freebsd.org <mailto:freebsd-java-unsubscribe at freebsd.org>"
>> 
>> 
> 
> 