Helping out with JDK 9 on BSD

[Magnus Ihse Bursie]

Resending. 

/Magnus
> On 2016-02-01 23:11, Magnus Ihse Bursie wrote:
> _______________________________________________
> freebsd-java at freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-java
> To unsubscribe, send any mail to "freebsd-java-unsubscribe at freebsd.org"

I'm not sure what happened there.

Here is what I sent:

> On 2016-02-01 16:46, Brian Gardner wrote:
> Hi Magnus,
> I’ve been looking into it further and it lead me to a couple of change sets:
> http://hg.openjdk.java.net/jdk9/hs-rt/hotspot/rev/469835cd5494
> and then this change set:
> http://hg.openjdk.java.net/jdk9/dev/hotspot/rev/f7dc8eebc3f5
> 
> The second change set titled,  Replace ThreadLocalStorage with compiler/language-based thread-local variables,
> seems to add in logic for doing compiler based thread local storage THREAD_LOCAL_DECL, but fails to actually replace the ThreadLocalStorage usage when persisting the current thread.  I couldn’t understand why you would want to persist both ways. 

I believe the reason for this is given in the bug description (https://bugs.openjdk.java.net/browse/JDK-8132510):
"The code underpinning __thread use is not async-signal-safe, which is not really a surprise as pthread_get/setspecific are not designated async-signal-safe either.

[...]

So I've reinstated a very basic ThreadLocalStorage class which will only need two implementations: a POSIX one, and a Windows one. This class is always initialized and ThreadLocalStorage::thread() is used from the signal handlers (as today). "

>  In any case it seemed harmless, and I’m trying to get gdb working with my images so I can debug the TLS issue further.  
> 
> It sounds like you already have an idea how to fix it though.  Any ideas or pointers would be greatly appreciated.

I think you've dug up about as much as I did. I believe the main cause for the problems is JDK-8132510 (the second of your changesets) which rewrote the entire infrastructure on how TLS is used in hotspot. And this was not tested on bsd. I know it took some iterations to get it right on the platforms it was tested on. I think one approach is to study this changeset more in detail on how linux and macosx is handled, which will likely give some hints on what's missing for bsd. It seems you've might have been finding some discrepancies already. 

The first of your changesets indicate a way this same problem was fixed previously. It seems to me that this fix (creating the key with a restore_thread_pointer callback) is implemented in JDK-8132510, but maybe it doesn't work. Perhaps there is some difference in the implementation of this on FreeBSD? From the manpage for pthread_key_create I read:

     If, after all the destructors have been called for all non-NULL values
     with associated destructors, there are still some non-NULL values with
     associated destructors, then the process is repeated.  If, after at least
     [PTHREAD_DESTRUCTOR_ITERATIONS] iterations of destructor calls for out-
     standing non-NULL values, there are still some non-NULL values with asso-
     ciated destructors, the implementation stops calling destructors.

It seems that PTHREAD_DESTRUCTOR_ITERATIONS is 4 on FreeBSD. Maybe     it's larger on other systems? (Checking on my Ubuntu, I read 4 there as well). Hm, so maybe something is iterating out of control here.


I have some ideas of it's cause, at least. I'll provide what info I have.

This was (probably) introduced by JDK-8132510 (the second of your changesets), which rewrote the entire infrastructure on how TLS is used in hotspot. And this was not adapted to bsd. I think one approach is to study this changeset more in detail on how linux and macosx is handled, which will likely give some hints on what's missing for bsd.

I'm currently trying to recompile with -DUSE_LIBRARY_BASED_TLS_ONLY, to see if that modifies the behavior. I don't really think it will, but if it does, it'll shed some light on the problem.

Another approach is to compile a debug build (configure --enable-debug) and see if this assert is triggered:
 assert(!_initialized, "initializing TLS more than once!");

/Magnus