main (14-CURRENT) may be unstable on powerpc64

Mark Millard marklmi at yahoo.com
Sun Feb 7 01:23:22 UTC 2021


On 2021-Feb-6, at 12:55, Mark Millard <marklmi at yahoo.com> wrote:

> On 2021-Feb-5, at 23:49, Mark Millard <marklmi at yahoo.com> wrote:
> 
>> On 2021-Feb-5, at 21:34, Mark Millard <marklmi at yahoo.com> wrote:
>> 
>>> On 2021-Feb-5, at 10:34, Mark Millard <marklmi at yahoo.com> wrote:
>>> 
>>> On 2021-Feb-5, at 09:35, Justin Hibbits <chmeeedalf at gmail.com> wrote:
>>>> 
>>>>> On Fri, 5 Feb 2021 04:05:55 -0800
>>>>> Mark Millard via freebsd-ppc <freebsd-ppc at freebsd.org> wrote:
>>>>> 
>>>>>> . . .
>>>>> 
>>>>> This is probably fallout from 710e45c4b, which has since been reverted.
>>>>> 710e45c4b broke other things like swig as well, which caused a lot of
>>>>> poudriere fallout for me (devel/llvm* failed because swig crashed).
>>>>> 
>>>>> Try updating past 33f0540b1 and testing again.
>>>>> 
>>>> 
>>>> The above is based on 847dfd2803f6, which is after 33f0540b1 . . .
>>>> 
>>>> https://cgit.freebsd.org/src/log/?qt=range&q=33f0540b1~1..847dfd2803f6
>>>> shows:
>>>> 
>>>> Commit message (Expand)	Author	Age	Files	Lines
>>>> * 	readelf: do not trucate section name with -W	Ed Maste	45 hours	1	-4/+9
>>>> * 	readelf: decode LA48 and ASG_DISABLE feature flags	Ed Maste	45 hours	1	-0/+2
>>>> * 	Add a VM flag to prevent reclaim on a failed contig allocation	Ryan Stone	45 hours	3	-2/+11
>>>> * 	dwmmc: Multiple busdma fixes.	Michal Meloun	46 hours	1	-15/+32
>>>> * 	linux: remove locks around callout_drain in timerfd_close()	shu	46 hours	1	-2/+0
>>>> * 	Revert "Reimplement strlen"	Mateusz Guzik	47 hours	2	-53/+108
>>>> 
>>>> (I Probably should have shown that in the original
>>>> message, given the difficulty in determining the
>>>> relative order of referenced commits.)
>>>> 
>>>> Before updating to be 847dfd2803f6 based, I had also
>>>> previously hit the swig issue with the llvm10 build.
>>>> That problem failed reliably until after I'd updated
>>>> past the revert. (Not trusting the other things built
>>>> is why I did a -c poudriere bulk after updating to an
>>>> environment based on after the revert.)
>>>> 
>>>> The variability in the pkg-static behavior this time
>>>> suggests race conditions are involved, though not
>>>> frequent failures.
>>>> 
>>>> 
>>> 
>>> It still failed to build llvm10 but the error reporoted
>>> is rather different. At least there is a .tbz for me to
>>> expand. I may end up with more to report after that.
>>> 
>>> Here is the log file's report:
>>> 
>>> [2996/4558] /usr/bin/c++ -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -Itools/clang/tools/extra/clang-tidy/plugin -I/wrkdirs/usr/ports/devel/llvm10/work/llvm-10.
>>> 0.1.src/tools/clang/tools/extra/clang-tidy/plugin -I/wrkdirs/usr/ports/devel/llvm10/work/llvm-10.0.1.src/tools/clang/include -Itools/clang/include -Iinclude -I/wrkdirs/usr/ports/devel/llvm10/work/llvm
>>> -10.0.1.src/include -O2 -pipe -DNDEBUG -fstack-protector-strong -isystem /usr/local/include -fno-strict-aliasing  -DNDEBUG -isystem /usr/local/include -fPIC -fvisibility-inlines-hidden -Werror=date-ti
>>> me -Werror=unguarded-availability-new -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wmissing-field-initializers -pedantic -Wno-long-long -Wimplicit-fallthrough -Wcovered-switch-defa
>>> ult -Wno-noexcept-type -Wnon-virtual-dtor -Wdelete-non-virtual-dtor -Wstring-conversion -fdiagnostics-color -ffunction-sections -fdata-sections -fno-common -Woverloaded-virtual -Wno-nested-anon-types 
>>> -O2 -pipe -DNDEBUG -fstack-protector-strong -isystem /usr/local/include -fno-strict-aliasing  -DNDEBUG -isystem /usr/local/include  -fno-exceptions -std=c++14 -MD -MT tools/clang/tools/extra/clang-tid
>>> y/plugin/CMakeFiles/obj.clangTidyPlugin.dir/ClangTidyPlugin.cpp.o -MF tools/clang/tools/extra/clang-tidy/plugin/CMakeFiles/obj.clangTidyPlugin.dir/ClangTidyPlugin.cpp.o.d -o tools/clang/tools/extra/cl
>>> ang-tidy/plugin/CMakeFiles/obj.clangTidyPlugin.dir/ClangTidyPlugin.cpp.o -c /wrkdirs/usr/ports/devel/llvm10/work/llvm-10.0.1.src/tools/clang/tools/extra/clang-tidy/plugin/ClangTidyPlugin.cpp
>>> FAILED: tools/clang/tools/extra/clang-tidy/plugin/CMakeFiles/obj.clangTidyPlugin.dir/ClangTidyPlugin.cpp.o 
>>> /usr/bin/c++ -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -Itools/clang/tools/extra/clang-tidy/plugin -I/wrkdirs/usr/ports/devel/llvm10/work/llvm-10.0.1.src/tool
>>> s/clang/tools/extra/clang-tidy/plugin -I/wrkdirs/usr/ports/devel/llvm10/work/llvm-10.0.1.src/tools/clang/include -Itools/clang/include -Iinclude -I/wrkdirs/usr/ports/devel/llvm10/work/llvm-10.0.1.src/
>>> include -O2 -pipe -DNDEBUG -fstack-protector-strong -isystem /usr/local/include -fno-strict-aliasing  -DNDEBUG -isystem /usr/local/include -fPIC -fvisibility-inlines-hidden -Werror=date-time -Werror=u
>>> nguarded-availability-new -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wmissing-field-initializers -pedantic -Wno-long-long -Wimplicit-fallthrough -Wcovered-switch-default -Wno-noe
>>> xcept-type -Wnon-virtual-dtor -Wdelete-non-virtual-dtor -Wstring-conversion -fdiagnostics-color -ffunction-sections -fdata-sections -fno-common -Woverloaded-virtual -Wno-nested-anon-types -O2 -pipe -D
>>> NDEBUG -fstack-protector-strong -isystem /usr/local/include -fno-strict-aliasing  -DNDEBUG -isystem /usr/local/include  -fno-exceptions -std=c++14 -MD -MT tools/clang/tools/extra/clang-tidy/plugin/CMa
>>> keFiles/obj.clangTidyPlugin.dir/ClangTidyPlugin.cpp.o -MF tools/clang/tools/extra/clang-tidy/plugin/CMakeFiles/obj.clangTidyPlugin.dir/ClangTidyPlugin.cpp.o.d -o tools/clang/tools/extra/clang-tidy/plu
>>> gin/CMakeFiles/obj.clangTidyPlugin.dir/ClangTidyPlugin.cpp.o -c /wrkdirs/usr/ports/devel/llvm10/work/llvm-10.0.1.src/tools/clang/tools/extra/clang-tidy/plugin/ClangTidyPlugin.cpp
>>> PLEASE submit a bug report to https://bugs.freebsd.org/submit/ and include the crash backtrace, preprocessed source, and associated run script.
>>> Stack dump:
>>> 0.      Program arguments: /usr/bin/c++ -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -Itools/clang/tools/extra/clang-tidy/plugin -I/wrkdirs/usr/ports/devel/llvm1
>>> 0/work/llvm-10.0.1.src/tools/clang/tools/extra/clang-tidy/plugin -I/wrkdirs/usr/ports/devel/llvm10/work/llvm-10.0.1.src/tools/clang/include -Itools/clang/include -Iinclude -I/wrkdirs/usr/ports/devel/llvm10/work/llvm-10.0.1.src/include -O2 -pipe -DNDEBUG -fstack-protector-strong -isystem /usr/local/include -fno-strict-aliasing -DNDEBUG -isystem /usr/local/include -fPIC -fvisibility-inlines-hidden -Werror=date-time -Werror=unguarded-availability-new -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wmissing-field-initializers -pedantic -Wno-long-long -Wimplicit-fallthrough -Wcovered-switch-default -Wno-noexcept-type -Wnon-virtual-dtor -Wdelete-non-virtual-dtor -Wstring-conversion -fdiagnostics-color -ffunction-sections -fdata-sections -fno-common -Woverloaded-virtual -Wno-nested-anon-types -O2 -pipe -DNDEBUG -fstack-protector-strong -isystem /usr/local/include -fno-strict-aliasing -DNDEBUG -isystem /usr/local/include -fno-exceptions -std=c++14 -MD -MT tools/clang/tools/extra/clang-tidy/plugin/CMakeFiles/obj.clangTidyPlugin.dir/ClangTidyPlugin.cpp.o -MF tools/clang/tools/extra/clang-tidy/plugin/CMakeFiles/obj.clangTidyPlugin.dir/ClangTidyPlugin.cpp.o.d -o tools/clang/tools/extra/clang-tidy/plugin/CMakeFiles/obj.clangTidyPlugin.dir/ClangTidyPlugin.cpp.o -c /wrkdirs/usr/ports/devel/llvm10/work/llvm-10.0.1.src/tools/clang/tools/extra/clang-tidy/plugin/ClangTidyPlugin.cpp 
>>> 1.      /wrkdirs/usr/ports/devel/llvm10/work/llvm-10.0.1.src/tools/clang/include/clang/Frontend/Utils.h:165:76: current parser token ')'
>>> 2.      /wrkdirs/usr/ports/devel/llvm10/work/llvm-10.0.1.src/tools/clang/include/clang/Frontend/Utils.h:40:1: parsing namespace 'clang'
>>> 3.      /wrkdirs/usr/ports/devel/llvm10/work/llvm-10.0.1.src/tools/clang/include/clang/Frontend/Utils.h:149:1: parsing struct/union/class body 'clang::ModuleDependencyCollector'
>>> 4.      /wrkdirs/usr/ports/devel/llvm10/work/llvm-10.0.1.src/tools/clang/include/clang/Frontend/Utils.h:165:47: parsing function body 'clang::ModuleDependencyCollector::insertSeen'
>>> 5.      /wrkdirs/usr/ports/devel/llvm10/work/llvm-10.0.1.src/tools/clang/include/clang/Frontend/Utils.h:165:47: in compound statement ('{}')
>>> 6.      /usr/include/c++/v1/utility:297:29: instantiating class definition 'std::__1::pair<llvm::StringMapIterator<llvm::NoneType>, bool>'
>>> 7.      /wrkdirs/usr/ports/devel/llvm10/work/llvm-10.0.1.src/include/llvm/ADT/StringMap.h:34:34: instantiating class definition 'llvm::StringMapIterator<llvm::NoneType>'
>>> 8.      /wrkdirs/usr/ports/devel/llvm10/work/llvm-10.0.1.src/include/llvm/ADT/StringMap.h:488:7: instantiating class definition 'llvm::StringMapIterBase<llvm::StringMapIterator<llvm::NoneType>, llvm::StringMapEntry<llvm::NoneType>>'
>>> 9.      /wrkdirs/usr/ports/devel/llvm10/work/llvm-10.0.1.src/include/llvm/ADT/iterator.h:67:7: instantiating class definition 'llvm::iterator_facade_base<llvm::StringMapIterator<llvm::NoneType>, std::__1::forward_iterator_tag, llvm::StringMapEntry<llvm::NoneType>, long, llvm::StringMapEntry<llvm::NoneType> *, llvm::StringMapEntry<llvm::NoneType> &>'
>>> #0 0x0000000012f52ed0 llvm::sys::PrintStackTrace(llvm::raw_ostream&) (/usr/bin/c+++0x12f52ed0)
>>> #1 0x0000000012f50770 llvm::sys::RunSignalHandlers() (/usr/bin/c+++0x12f50770)
>>> #2 0x0000000012ee4778 CrashRecoverySignalHandler(int) (/usr/bin/c+++0x12ee4778)
>>> #3 0x0000000813f2b2b4 (/lib/libthr.so.3+0x2f2b4)
>>> c++: error: clang frontend command failed due to signal (use -v to see invocation)
>>> FreeBSD clang version 11.0.1 (git at github.com:llvm/llvm-project.git llvmorg-11.0.1-0-g43ff75f2c3fe)
>>> Target: powerpc64-unknown-freebsd14.0
>>> Thread model: posix
>>> InstalledDir: /usr/bin
>>> c++: note: diagnostic msg: 
>>> ********************
>>> 
>>> PLEASE ATTACH THE FOLLOWING FILES TO THE BUG REPORT:
>>> Preprocessed source(s) and associated run script(s) are located at:
>>> c++: note: diagnostic msg: /tmp/ClangTidyPlugin-21aaa2.cpp
>>> c++: note: diagnostic msg: /tmp/ClangTidyPlugin-21aaa2.sh
>>> c++: note: diagnostic msg: 
>>> 
>>> 
>>> Note: Unfortunately, poudriere does not capture those files
>>> from /tmp/ in the .tbz for the build failure.
>>> 
>> 
>> No *.core was left behind in the expanded .tbz .
>> /var/log/messages did not have a line reporting
>> any pid signal lines in the time frame.
>> 
>> Repeating the failing command in the expansion of
>> the .tbz did not repeat the problem.
>> 
>> It looks to be another example of there being one
>> or more not-readily-repeatable problems occurring.
>> 
>> It will be many hours before the new poudriere bulk
>> can get back to that same point, but with just the
>> one builder and the one job for the builder. (Still
>> allowing use of an active process per cpu.) The
>> normal/prior bulk runs were allowing 2 jobs.
>> 
>> 
>> Note: /lib/libthr.so.3+0x2f2b4 seems to be in code
>> from the "static void handle_signal . . ." routine.
>> 
> 
> The devel/llvm10 rebuild attempt is well past the
> [2996/4558] point now: working on [3056/4558] .
> But it looks like it may have 6+ hrs to go to
> complete (if successful). So far:
> 
> [FBSDpowerpc64-default] [2021-02-05_23h31m51s] [parallel_build:] Queued: 1  Built: 0  Failed: 0  Skipped: 0  Ignored: 0  Tobuild: 1   Time: 13:01:28
>        [01]: devel/llvm10              | llvm10-10.0.1_3           build           (12:50:05 / 12:59:42)
> [13:02:12] Logs: /usr/local/poudriere/data/logs/bulk/FBSDpowerpc64-default/2021-02-05_23h31m51s
> 
> I've no reasonable way to figure out if the parallel
> activity from letting 2 jobs run (so: higher load
> average) would have made a difference or not. Getting
> a very detailed repetition for such is difficult
> so any variation in what context it failed in makes
> useful comparison/contrast problematical.
> 
> But, if this run completes successfully, there will
> be 101 more ports to let poudriere bulk try to build,
> one of which would be devel/llvm11 .
> 

It finished, apparently successfully:

[17:36:24] Built ports: devel/llvm10
[FBSDpowerpc64-default] [2021-02-05_23h31m51s] [committing:] Queued: 1  Built: 1  Failed: 0  Skipped: 0  Ignored: 0  Tobuild: 0   Time: 17:35:40
[17:36:24] Logs: /usr/local/poudriere/data/logs/bulk/FBSDpowerpc64-default/2021-02-05_23h31m51s

I've started the attempt to build what is still pending:

. . .
[FBSDpowerpc64-default] [2021-02-06_17h13m39s] [balancing_pool:] Queued: 101 Built: 0   Failed: 0   Skipped: 0   Ignored: 0   Tobuild: 101  Time: 00:02:44
[00:03:27] Recording filesystem state for prepkg... done
[00:03:27] Building 101 packages using 2 builders
[00:03:27] Starting/Cloning builders
[00:04:15] Hit CTRL+t at any time to see build progress and stats
[00:04:15] [01] [00:00:00] Building graphics/mesa-libs | mesa-libs-20.2.3
[00:04:15] [02] [00:00:00] Building devel/llvm11 | llvm11-11.0.0
. . .

===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)



More information about the freebsd-ppc mailing list