Helping out with JDK 9 on BSD
Brian Gardner
openjdk at getsnappy.com
Thu Feb 11 06:46:40 UTC 2016
Hello again Greg and Magnus,
I finished up researching the leftover thread-specific data after 4 destructor iterations issue. It looks like it’s currently functioning on Freebsd the same way it does on Linux, the only difference is that Freebsd output that warning were Linux silently stops after it’s forth try. I looked into how openjdk8 was working and it looks like there were calls to TLS::set_thread(NULL) scattered throughout the code base at the end of some ::run methods. I’ve mimicked that behavior in the necessary spots with the following patch. I can’t help but think there must be a better way to get these Thread’s destructors called, and I tried that in place of making clear_thread_current public and calling it directly, but it resulted in deadlock.
> On Feb 9, 2016, at 2:36 PM, Brian Gardner <openjdk at getsnappy.com> wrote:
>
> Hi Greg and Magnus,
> I’ve attached the patches needed to build and compile on Freebsd. I’ve reverted my changes to TLS, and done more debugging there. There appears to be two separate issues. One issue was that the implementation of OS::bad::gettid() was somehow clearing the current thread previously set with pthread_setspecific. I was able to fix this by porting logic from bad-port/jdk8, see hotspot__os_bd_cpp__getthreadid.patch. The second issue seems to be related to certain types of threads never getting their destructor called and therefor not clearing pthread_setspecific and resulting in Thread 803c59000 has exited with leftover thread-specific data after 4 destructor iterations, errors. This one I haven’t tracked down yet and will continue looking into. While I can compile and execute HelloWorld, I’m getting a ton of these benign yet annoying messages from ConcurrentGC and Worker threads. I’ll continue looking into that next week.
>
>
>> On Feb 4, 2016, at 11:50 PM, Magnus Ihse Bursie <magnus.ihse.bursie at oracle.com <mailto:magnus.ihse.bursie at oracle.com>> wrote:
>>
>> On 2016-02-04 01:29, Brian Gardner wrote:
>>> Hi Greg,
>>> Great to hear from you, it’s been a while since I took on one of these projects, but I do follow the java at freebsd mailing list and see all the work you put in. I must tell you how greatly your work is appreciated by me and I’m sure the rest of the community. I have good news, with fairly minimal changes I have it compiling, building HelloWorld, and running HelloWorld all without errors.
>>>
>>> I think it’ll break down into 4 change sets,
>>> NM - I ported some code from bsd-port/jdk8/hotspot/make/bsd/makefiles/build_vm_def.sh that fixes a compile error
>>> SUPPORT_RESERVED_STACK_AREA - define SUPPORT_RESERVED_STACK_AREA for _ALLBSD_SOURCE
>>> Serviceability Agent - ported from bad-port/jdk8, currently compiles but hasn’t been tested
>>> TLS - there is problems with the TLS in jdk9 used for Thread::current(). This caused several issues for me and while the fix was relatively straight forward in using THREAD_LOCAL_DECL completely in place of TLS completely, I’d imagine there was a reason it was implemented in sort of a Hybrid fashion alongside TLS.
>>
>> There is a reason. It is very well described in this comment:
>> https://bugs.openjdk.java.net/browse/JDK-8132510?focusedCommentId=13866505 <https://bugs.openjdk.java.net/browse/JDK-8132510?focusedCommentId=13866505> <https://bugs.openjdk.java.net/browse/JDK-8132510?focusedCommentId=13866505 <https://bugs.openjdk.java.net/browse/JDK-8132510?focusedCommentId=13866505>>
>>
>> The TL;DR: in some circumstances the JVM can crash if we do not set up an initial TLS. On the other hand, this was for glibc, so maybe this situation does not arise for FreeBSD.
>>
>>>
>>> It would be great if I could post these as web reviews for others to review. Greg, would getting access to bsd-port/jdk9 allow us to collaborate through web reviews?
>>
>>
>>> Magnus, do you think using bsd-port/jdk9 for this purpose be a step towards getting these basic changes into the jdk9 mainline, or would your colleagues think "if they have bsd-port/jdk9, why bother with integrating them into the jdk9 mainline?”
>>
>> I think putting these changes in bsd-port/jdk9 is very reasonable, and I do not believe it will in any way hinder their acceptance into jdk9 mainline. The only thing to be careful about there, though, is that all contributors have signed the OCA (Oracle contributor agreement). Mixing in "legally bad" code can present a real problem for adoption into mainline.
>>
>> /Magnus
>>
>>>
>>> Brian Gardner
>>>
>>>> On Feb 2, 2016, at 7:40 PM, Greg Lewis < <mailto:glewis at eyesbeyond.com <mailto:glewis at eyesbeyond.com>>glewis at eyesbeyond.com <mailto:glewis at eyesbeyond.com> <mailto:glewis at eyesbeyond.com <mailto:glewis at eyesbeyond.com>>> wrote:
>>>>
>>>> On Sun, Jan 31, 2016 at 06:57:26AM -0800, Brian Gardner wrote:
>>>>> I???m interested in helping. I ported openjdk6 to freebsd and also helped out with openjdk8.
>>>>>
>>>>> Brian Gardner
>>>>
>>>> I'd love to see as much of these changes get into the jdk9 mainline. My time
>>>> is really limited for the next couple of weeks though. If it would help you
>>>> or Magnus to use the bsd-port repo to stage changes then that would be
>>>> great too. Let me do a merge to make sure it is up to date with mainline.
>>>> Notionally we'd have to do a vote to get you both access I think? But I
>>>> can shepherd changes in if that will help with sharing them.
>>>>
>>>> FWIW, in terms of taking responsibility, I've been regularly updating the
>>>> repos for jdk8 and jdk7 for a couple of years. I'm happy to keep jdk9
>>>> building if that is what is being looked for there and also port more of
>>>> the BSD changes for jdk8 as appropriate.
>>>>
>>>> - Greg
>>>>
>>>>>> On Jan 31, 2016, at 3:00 AM, Magnus Ihse Bursie <magnus.ihse.bursie at oracle.com <mailto:magnus.ihse.bursie at oracle.com> <mailto:magnus.ihse.bursie at oracle.com <mailto:magnus.ihse.bursie at oracle.com>>> wrote:
>>>>>>
>>>>>> On 2016-01-31 11:54, Magnus Ihse Bursie wrote:
>>>>>>> Hi,
>>>>>>>
>>>>>>> [TL;DR: I'm offering patches to compile JDK 9 on FreeBSD, but it's not clear where to put them, or how.]
>>>>>>>
>>>>>>> I'm working at Oracle on the OpenJDK build team and is responsible for large parts of the build system of OpenJDK. Lately, I've been playing around with FreeBSD (and other BSDs) in my free time, and I've written a patch that will add build system support for FreeBSD, OpenBSD and NetBSD in JDK 9 (tracked in <https://bugs.openjdk.java.net/browse/JDK-8147795 <https://bugs.openjdk.java.net/browse/JDK-8147795>>https://bugs.openjdk.java.net/browse/JDK-8147795 <https://bugs.openjdk.java.net/browse/JDK-8147795><https://bugs.openjdk.java.net/browse/JDK-8147795 <https://bugs.openjdk.java.net/browse/JDK-8147795>>).
>>>>>>>
>>>>>>> I started writing this when I realized that the jdk9 branch in the OpenJDK bsd-port repository did not contain any BSD-specific changes at all, and the old JDK 8 changes will not readily be portable, due to major changes in the build system between JDK 8 and JDK 9.
>>>>>>>
>>>>>>> I thought it would be a no-brainer to integrate these changes into the JDK 9 mainline, so they would be in place for whenever you guys would start to attack porting the code base. However, some of my collegues thought otherwise. The end result, I think, is that they wanted to see someone (not necessarily a company, the FreeBSD organisation for instance seemed to be okay), to step forward and say "we take responsibility for the BSD port", and give some kind of commitment to actually use these build changes in producing a viable port. You can read the mail conversation here: <http://mail.openjdk.java.net/pipermail/build-dev/2016-January/016421.html <http://mail.openjdk.java.net/pipermail/build-dev/2016-January/016421.html>>http://mail.openjdk.java.net/pipermail/build-dev/2016-January/016421.html <http://mail.openjdk.java.net/pipermail/build-dev/2016-January/016421.html> <http://mail.openjdk.java.net/pipermail/build-dev/2016-January/016421.html <http://mail.openjdk.java.net/pipermail/build-dev/2016-January/016421.html>>
>>>>>>
>>>>>> I forgot to add that I'm willing to help with build issues for the BSD port, as far as my (like everybode else's) limited free time allows me. Unfortunately, I'm still a noob at BSD internals and can't really help with much apart from that.
>>>>>>
>>>>>> /Magnus
>>>>>>
>>>>>>
>>>>>>>
>>>>>>> An alternative to pushing this code into the JDK 9 mainline is of course to push it to the bsd-port/jdk9 repo (given that the owners of that repo approves), but that seems like a less favourable solution. Having the code in the mainline does not mean that it gets tested automatically, but it means that it will be part of e.g. refactoring, that would otherwise break a downstream patchset.
>>>>>>>
>>>>>>> I cc:ed this conversation to the <mailto:bsd-port-dev at openjdk.java.net <mailto:bsd-port-dev at openjdk.java.net>>bsd-port-dev at openjdk.java.net <mailto:bsd-port-dev at openjdk.java.net> <mailto:bsd-port-dev at openjdk.java.net <mailto:bsd-port-dev at openjdk.java.net>> mailing list, but never got any kind of official response there. Since this list seems more active, I'm trying here instead. :)
>>>>>>>
>>>>>>> So, I'm offering two patches here, one that applies to the build system, is nice and clean, and possible to integrate into JDK 9 mainline, if my collegues are convinced that someone is backing up the BSD port. And there's a second patch, which fixes broken C/C++/Java code and results in a product that can at least run "javac HelloWorld", but this is not yet clean enough for integration anywere, at least not the JDK 9 mainline. (I don't know enough of the BSD internals to fix all problems, so there's some "#if 0" code here and there.)
>>>>>>>
>>>>>>> /Magnus
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> freebsd-java at freebsd.org <mailto:freebsd-java at freebsd.org> <mailto:freebsd-java at freebsd.org <mailto:freebsd-java at freebsd.org>> mailing list
>>>>>>> https://lists.freebsd.org/mailman/listinfo/freebsd-java <https://lists.freebsd.org/mailman/listinfo/freebsd-java> <https://lists.freebsd.org/mailman/listinfo/freebsd-java <https://lists.freebsd.org/mailman/listinfo/freebsd-java>>
>>>>>>> To unsubscribe, send any mail to "freebsd-java-unsubscribe at freebsd.org <mailto:freebsd-java-unsubscribe at freebsd.org>" <mailto:freebsd-java-unsubscribe at freebsd.org <mailto:freebsd-java-unsubscribe at freebsd.org>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> freebsd-java at freebsd.org <mailto:freebsd-java at freebsd.org> <mailto:freebsd-java at freebsd.org <mailto:freebsd-java at freebsd.org>> <mailto:freebsd-java at freebsd.org <mailto:freebsd-java at freebsd.org> <mailto:freebsd-java at freebsd.org <mailto:freebsd-java at freebsd.org>>> mailing list
>>>>>> https://lists.freebsd.org/mailman/listinfo/freebsd-java <https://lists.freebsd.org/mailman/listinfo/freebsd-java> <https://lists.freebsd.org/mailman/listinfo/freebsd-java <https://lists.freebsd.org/mailman/listinfo/freebsd-java>> <https://lists.freebsd.org/mailman/listinfo/freebsd-java <https://lists.freebsd.org/mailman/listinfo/freebsd-java> <https://lists.freebsd.org/mailman/listinfo/freebsd-java <https://lists.freebsd.org/mailman/listinfo/freebsd-java>>>
>>>>>> To unsubscribe, send any mail to " <mailto:freebsd-java-unsubscribe at freebsd.org <mailto:freebsd-java-unsubscribe at freebsd.org>>freebsd-java-unsubscribe at freebsd.org <mailto:freebsd-java-unsubscribe at freebsd.org> <mailto:freebsd-java-unsubscribe at freebsd.org <mailto:freebsd-java-unsubscribe at freebsd.org>> < <mailto:freebsd-java-unsubscribe at freebsd.org <mailto:freebsd-java-unsubscribe at freebsd.org>>mailto:freebsd-java-unsubscribe at freebsd.org <mailto:freebsd-java-unsubscribe at freebsd.org> <mailto:freebsd-java-unsubscribe at freebsd.org <mailto:freebsd-java-unsubscribe at freebsd.org>>>"
>>>>>
>>>>> _______________________________________________
>>>>> freebsd-java at freebsd.org <mailto:freebsd-java at freebsd.org> <mailto:freebsd-java at freebsd.org <mailto:freebsd-java at freebsd.org>> mailing list
>>>>> https://lists.freebsd.org/mailman/listinfo/freebsd-java <https://lists.freebsd.org/mailman/listinfo/freebsd-java> <https://lists.freebsd.org/mailman/listinfo/freebsd-java <https://lists.freebsd.org/mailman/listinfo/freebsd-java>>
>>>>> To unsubscribe, send any mail to "freebsd-java-unsubscribe at freebsd.org <mailto:freebsd-java-unsubscribe at freebsd.org>" <mailto:freebsd-java-unsubscribe at freebsd.org <mailto:freebsd-java-unsubscribe at freebsd.org>>
>>>>
>>>> --
>>>> Greg Lewis Email : <mailto:glewis at eyesbeyond.com <mailto:glewis at eyesbeyond.com>>glewis at eyesbeyond.com <mailto:glewis at eyesbeyond.com> <mailto:glewis at eyesbeyond.com <mailto:glewis at eyesbeyond.com>>
>>>> Eyes Beyond Web : <http://www.eyesbeyond.com/ <http://www.eyesbeyond.com/>>http://www.eyesbeyond.com <http://www.eyesbeyond.com/> <http://www.eyesbeyond.com/ <http://www.eyesbeyond.com/>>
>>>> Information Technology FreeBSD : <mailto:glewis at freebsd.org <mailto:glewis at freebsd.org>>glewis at FreeBSD.org <mailto:glewis at freebsd.org> <mailto:glewis at FreeBSD.org <mailto:glewis at FreeBSD.org>>
>>>> _______________________________________________
>>>> freebsd-java at freebsd.org <mailto:freebsd-java at freebsd.org> <mailto:freebsd-java at freebsd.org <mailto:freebsd-java at freebsd.org>> mailing list
>>>> https://lists.freebsd.org/mailman/listinfo/freebsd-java <https://lists.freebsd.org/mailman/listinfo/freebsd-java> <https://lists.freebsd.org/mailman/listinfo/freebsd-java <https://lists.freebsd.org/mailman/listinfo/freebsd-java>>
>>>> To unsubscribe, send any mail to "freebsd-java-unsubscribe at freebsd.org <mailto:freebsd-java-unsubscribe at freebsd.org> <mailto:freebsd-java-unsubscribe at freebsd.org <mailto:freebsd-java-unsubscribe at freebsd.org>>"
>>>
>>
>>
>
> _______________________________________________
> freebsd-java at freebsd.org <mailto:freebsd-java at freebsd.org> mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-java <https://lists.freebsd.org/mailman/listinfo/freebsd-java>
> To unsubscribe, send any mail to "freebsd-java-unsubscribe at freebsd.org <mailto:freebsd-java-unsubscribe at freebsd.org>"
More information about the freebsd-java
mailing list