mysterious hang in pthread_create

[Kostik Belousov]

On Sun, Aug 31, 2008 at 11:39:52AM -0400, Daniel Eischen wrote:
> On Sat, 30 Aug 2008, Kostik Belousov wrote:
> 
> >On Sat, Aug 30, 2008 at 12:15:31PM -0400, Daniel Eischen wrote:
> >>On Sat, 30 Aug 2008, Kostik Belousov wrote:
> >>
> >>>On Sat, Aug 30, 2008 at 11:32:35AM -0400, Daniel Eischen wrote:
> >>>>On Fri, 29 Aug 2008, Kostik Belousov wrote:
> >>>>>
> >>>>>As demonstrated by Andriy' example, we need _thr_rtld_init() be called
> >>>>>before any rtld locks are given chance to be acquired. _thr_rtld_init()
> >>>>>shall be protected from repeated invocation, and _thr_setthreaded()
> >>>>>implements exactly this.
> >>>>>
> >>>>>If calling _thr_setthreaded(1) has not quite right intent, could you,
> >>>>>please, suggest satisfying solution ?
> >>>>
> >>>>I'm not sure I _quite_ understand the problem, but why
> >>>>wouldn't you have the same potential problem with some
> >>>>other library (without libthread)?  I'll have to go back
> >>>>and read the beginning of the thread - I just kinda came
> >>>>in at the end.
> >>>
> >>>Sure, for appropriate value of any. If you mean whether the same problem
> >>>would arise for any threading library that supplies locking 
> >>>implementation
> >>>for rtld, then certainly yes. I looked over and patched only libthr
> >>>since this is the only survived library for now.
> >>
> >>What I mean is, is fixing libthr a solution that will work
> >>for cases?  Or, is libthr doing something wrong?  I can't
> >>really see that it is.
> >>
> >>libthr assumes that everything is single-threaded (or
> >>serialized, I guess) before a thread is created.  I
> >>am looking at this thread:
> >>
> >>  http://docs.freebsd.org/cgi/getmsg.cgi?fetch=5235+0+current/freebsd-threads
> >>
> >>Where is the corresponding unlock for the wlock_acquire()?
> >>I guess this is the problem.  When would this normally
> >>be released (without libthr being linked in)?
> >>
> >>Also, the __isthreaded flag is used in libc to avoid taking
> >>locks unless necessary.  So if you have a single threaded
> >>application that is also linked with libthr, you don't
> >>pay the penalty of locking overhead.  Lots of 3rd-party
> >>libraries link with a threads library, so an application
> >>may not even know it is "threaded".
> >>
> >>>
> >>>Anyway, I do not insist on the proposed solution, and definitely
> >>>prefer the change that is well aligned with libthr architecture.
> >>
> >>I'm not arguing anything, I just don't know that the problem
> >>lies within lib<insert thread library here>.  Of course, the
> >>rtld init stuff could be pulled out and done in thread
> >>initialization instead of thr_setthreaded().  That doesn't
> >>leave much in thr_setthreaded, and it also adds locking
> >>overhead into rtld for single-threaded programs that are
> >>linked with libthr...
> >
> >Ok, let me to tell the whole story. I am sure that in fact you know
> >it better then me.
> >
> >Assuming libthr is the only threading library, there are two locking
> >implementations for the rtld: 'default' and the one supplied by libthr.
> >On the first call to pthread_create(), libthr calls _rtld_thread_init()
> >to substitute the default by the implementation from libthr.
> >
> >In fact, default implementation is broken from my point of view. For
> >instance, thread_flag update is not atomic. Moreover, it does not
> >correctly handles sequential acquision of several locks, due
> >to thread_flag.
> >
> >The dl_iterate_phdr() function, called by gcc exception handling support
> >code, does exactly this. It acquires rtld_phdr_lock, then rtld_bind_lock.
> >[I shall admit it does this after my change]. In particular, this would
> >leave the bit for the bind lock set in the thread_flag.
> >
> >Andriy' example throw the exception and calls dl_iterate_phdr() before
> >first thread is created. On thread creation, _rtld_thread_init() is
> >called, that tries to move the locks according to thread_flag. This is
> >the cause for the reported wlock acquisition.
> >
> >I do not want to change anything in the default rtld locking. It is
> >disfunctional from the time libc_r is gone, and I think it would be
> >better to make it nop. My change makes the image that is linked with
> >libthr, to consistently use libthr locks.
> 
> What happens if you remove the thread_flag() stuff (support
> for libc_r?) from rtld?  It seems that libc_r should be providing
> its own rtld locking hooks - just like libthr does.

No, libc_r uses default rtld locking, as absence of the references to
_rtld_thread_init in libc_r code indicates. This is one of the reasons
why I decline to change the code. We still support compat-4.x on all
supported systems, but I have no way to properly test it,

If changing the default implementation, I think we should just record
lock attempts to be able to transfer them when _rtld_thread_init is
called. But this would break libc_r.

I saw the use of the libthr locking from the start as the cleanest and
least intrusive. As I said, I would gladly accept any better idea.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 195 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-threads/attachments/20080831/216801a4/attachment.pgp