start building thread support for lang/perl5.12 by default

[John Hein]

Tom Hukins wrote at 16:19 +0100 on Oct 21, 2010:
 > On Thu, Oct 21, 2010 at 04:52:20PM +0200, Emanuel Haupt wrote:
 > > May I suggest to start enabling thread support for lang/perl5.12 by
 > > default. While thread support was considered unstable for a long time
 > > it has become more stable with perl 5.12.
 > 
 > I suggest we keep the default of not supporting threaded applications.

I've been wanting to revive this, err... thread for a while. [1]

I would like to suggest that we at least link perl with -pthread even
if we decide to not make perl itself threaded.  Currently the knobs in
the perl port Makefile don't support that option.  WITH_THREADS=yes
currently actually builds perl turning on threads code in perl itself,
in addition to linking with -pthread (${PTHREAD_LIBS}).

I'll explain why I think we should link with -pthread by default
below, but first address some of Tom's comments.  [3]


 > The stability of threads doesn't matter: applications written in Perl
 > either need them or they don't.  If an application doesn't need
 > threads it won't encounter thread-related instability.  If an
 > application needs threads, it needs them.

Agreed, assuming linking with -pthread has no other side effects.  As
it turns out it does have other side effects [2], but I think we still
should link with -pthread by default [3].


 > Any perl binary built with threading support runs slower than a
 > comparable binary without threading support regardless
 > of whether an application uses threads or not.  Shipping a
 > thread-enabled perl by default will cause users' applications to run
 > slower by default.

For this reason at least, I believe we should separate the
WITH_THREADS option into two:
 - build a threaded perl (configures perl with -Dusethreads=y) [default: off]
 - link with -pthread [default: on]

 > Specific applications or ports that need threads can install a
 > threaded perl port or package.

The ports system for perl on FreeBSD is currently not set up to
support two versions of perl.  If you have two different perl
applications that must coexist on one system, one that wants perl
WITH_THREADS=yes and the other perl WITHOUT_THREADS=yes, this policy
will not work.  In most mundane perl apps, that is an unlikely
conflict, but I have definitely encountered it. [2]

It's possible, but somewhat hard with the current ports setup, to have
two versions of perl.  Because of that, we should pick as the default
flavor the version of perl that will work with most apps.

That has been what guided the policy to _not_ use perl with -pthread
by default.  But perhaps these days the stack size problem affects
less ports than the perl without -pthread.  Unfortunately, I don't
have a very complete survey, but see [2]


 > > Finally, many Linux distributions have started to ship perl with thread
 > > support by default (eg. RHEL5).
 > 
 > We use RedHat at work and build our own perl binary largely to avoid
 > the slowness of threaded perls.  We also build with PERL_DISABLE_PMC
 > and NO_MATHOMS for further performance improvements.  One of my
 > colleagues is an ex-pumpking and has benchmarked these although he
 > doesn't have any numbers to hand.

Just curious, is that unthreaded perl or perl not linked with threading
library or both?


[1] some previous email threads:

amanda hangs when invokes modules that create threads, and Anton
Berezin's reply:
http://lists.freebsd.org/pipermail/freebsd-perl/2010-March/002511.html

stack size problems with -pthread:
http://lists.freebsd.org/pipermail/freebsd-hackers/2008-January/022916.html
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6293
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6060

There was another thread on threads@, I think, that described
the tradeoffs and the decision to limit the stack size when linking
with the threading library.  If I find it, I'll send a pointer.


[2]
 Just a couple applications that need perl and have conflicting
 issues with perl linked with -pthread or not:

 spamassassin (and amavisd which calls spamassassin) wants a perl that
 is not linked with -pthread because the default stack size once you
 link with -pthread is too small (bus errors, seg faults).  This may
 no longer be an issue with spamassassin as it was large regex
 expressions on the stack that caused problems.  Apparently perl 5.10
 uses the heap for regexes now.

 amanda (currently server apps only, but soon clients may have
 threaded code called via perl) wants perl linked with -pthread,
 else it hangs when the perl modules try to start threads.


[3]
I think we should turn on "link with -pthread" by default.
This will support the growing list of apps with perl modules that
use threads.

Then any apps that will break due to stack size too small (likely few,
but that's not much more than a guess) get patched to call
pthread_attr_setstacksize(3), possibly via wrapper, to increase the
stack size.  They aren't threaded anyway, so increasing the stack size
for the main (only) thread should be fine.


Below is a URL that has a tarball containing a simple test that Dustin
Mitchell devised that shows the problem.  It just tries to create some
threads in C code called via a perl module.

If you run it with a perl not linked with -pthread it will hang.
If you run it with a perl that _is_ linked with -pthread it should work.

Untar it and run it like so:
 perl Makefile.PL && make && env PERLLIB=${cwd}/lib:${cwd}/blib/arch perl -mMytest -e "Mytest::hello()"

http://filebin.ca/kacaxb/Mytest-0.01.tar.gz

It's attached, too, but that may not make it past the mailman
attachment stripper...