Migrating from -current to stable/12 on RPI2B (ARMv7)
Mark Millard
marklmi at yahoo.com
Tue Dec 29 00:14:59 UTC 2020
[I get the problem as well! I report a backtrace of the failure
and some more.]
On 2020-Dec-28, at 14:04, Mark Millard <marklmi at yahoo.com> wrote:
> On 2020-Dec-28, at 13:19, Mark Millard <marklmi at yahoo.com> wrote:
>
>> On 2020-Dec-28, at 12:55, Mark Millard <marklmi at yahoo.com> wrote:
>>>
>>>
>>> On 2020-Dec-28, at 12:07, Mark Millard <marklmi at yahoo.com> wrote:
>>>
>>>
>>>> On 2020-Dec-28, at 10:56, bob prohaska <fbsd at www.zefox.net> wrote:
>>>>
>>>>> On Sun, Dec 27, 2020 at 10:10:18PM -0800, Mark Millard wrote:
>>>>>>
>>>>>>
>>>>>> On 2020-Dec-27, at 20:48, bob prohaska <fbsd at www.zefox.net> wrote:
>>>>>>
>>>>>>> . . .
>>>>>>
>>>>> I didn't know about LDFLAGS, but a re-try with
>>>>> -j1 and LDFLAGS.lld+= -Wl,--threads=1
>>>>> (apparently the syntax changed) in /etc/make.conf
>>>>> promptly reproduced the error.
>>>>
>>>> Hmm. It been a while since I did a native build instead of a
>>>> cross build. The cross build context has RAM and does not
>>>> use the assignment so I'd not noticed.
>>>>
>>>> Thanks for the report!
>>>
>>> lld for LLVM 10 always had --no-threads as I now understand
>>> and stable/12 still has/uses/needs LLVM 10.0.1 (with
>>> updates).
>>>
>>> That means that lld from LLVM 11 was in use (FreeBSD
>>> 13's system ld). The build was probably trying to build
>>> some LLVM 10.0.1 final+ materials for bootstrap style
>>> build use in later build stages (older FreeBSD targeting).
>>> It likely had not gotten to the stage of building freebsd
>>> stable/12 material itself.
>>>
>>> Attempting to build devel/llvm10 might well have the same
>>> issue without having to involve an extra FreeBSD source
>>> tree or build.
>>
>> There is another gotcha-issue with the change from --no-threads
>> to --threads=1 based on neither working for both 10.0.x and
>> 11.0.y: which ever one is listed in /etc/make.conf (say) will be
>> wrong for other one of:
>>
>> A) building the bootstrap toolchain
>> B) using the bootstrap toolchain
>>
>> The LDFLAGS.lld definition would need to be conditional on the
>> distinction in order to be correct inside each type of context.
>
> I've started an experiment going another direction: an armv7
> context with lots of RAM (and faster processing) with a
> 13 attempting to build a stable/12 . It is via a chroot into
> an armv7 13 world on a 8 GiByte, 4-core Cortex-A57 based
> OverDrive 1000 running aarch64 13. Each process should still
> be limited to what 32-bit systems allow but overall the
> system is not that limited. I used -j4 .
>
> If such still got the error, then there would likely be
> implications about the error and it would be unlikely
> that it would work on the RPi2 v1.1 . (But I expect that
> it will not get the error.)
>
> Root owns file system involved and is doing the build.
>
> It did report:
>
> make[1]: "/usr/fbsd/stable-12-src/Makefile.inc1" line 344: SYSTEM_COMPILER: libclang will be built for bootstrapping a cross-compiler.
> make[1]: "/usr/fbsd/stable-12-src/Makefile.inc1" line 349: SYSTEM_LINKER: libclang will be built for bootstrapping a cross-linker.
>
> I already had a stable-12-src branch from git experiments
> but the matching worktree was missing despite being still
> registered. So the worktree recreation was:
>
> # git worktree add -f ../stable-12-src stable-12-src
> Preparing worktree (checking out 'stable-12-src')
> Updating files: 100% (81363/81363), done.
> HEAD is now at f4d0bc6aa6b9 MFC r354991-r354992 (by lwhsu)
For armv7 13's context being based on:
WITH_ASSERT_DEBUG=
WITH_LLVM_ASSERTIONS=
WITHOUT_MALLOC_PRODUCTION=
WITH_DEBUG_FILES=
-mcpu=cortex-a7
-g in use despite -O2 or such optimizations.
(The WITHOUT_MALLOC_PRODUCTION is because of a typo
in the attempt to have set WITH_MALLOC_PRODUCTION.)
I was not using --threads=1 since it would fail for later
build stages.
The being-built context was based on:
WITH_ASSERT_DEBUG=
WITHOUT_LLVM_ASSERTIONS=
WITH_MALLOC_PRODUCTION=
WITH_DEBUG_FILES=
-mcpu=cortex-a7
-g in use despite -O2 or such optimizations.
I got what follows. (But note that the ARM64TODO: fill_fpregs32c++
is likely from the handling of the process crash already in
process and is not contributing to the problem starting.)
. . .
--- all_subdir_usr.bin/clang/clang ---
[Creating objdir /usr/obj/rpi2_clang/arm.armv7/usr/fbsd/stable-12-src/arm.armv7/tmp/obj-tools/usr.bin/clang/clang...]
Building /usr/obj/rpi2_clang/arm.armv7/usr/fbsd/stable-12-src/arm.armv7/tmp/obj-tools/usr.bin/clang/clang/cc1_main.o
Building /usr/obj/rpi2_clang/arm.armv7/usr/fbsd/stable-12-src/arm.armv7/tmp/obj-tools/usr.bin/clang/clang/cc1as_main.o
Building /usr/obj/rpi2_clang/arm.armv7/usr/fbsd/stable-12-src/arm.armv7/tmp/obj-tools/usr.bin/clang/clang/cc1gen_reproducer_main.o
Building /usr/obj/rpi2_clang/arm.armv7/usr/fbsd/stable-12-src/arm.armv7/tmp/obj-tools/usr.bin/clang/clang/driver.o
Building /usr/obj/rpi2_clang/arm.armv7/usr/fbsd/stable-12-src/arm.armv7/tmp/obj-tools/usr.bin/clang/clang/clang.full
--- clang.full ---
LLVM ERROR: out of memory
PLEASE submit a bug report to https://bugs.freebsd.org/submit/ and include the crash backtrace.
Stack dump:
0. Program arguments: /usr/bin/ld --eh-frame-hdr -Bstatic -o clang.full /usr/lib/crt1.o /usr/lib/crti.o /usr/lib/crtbeginT.o -L/usr/obj/rpi2_clang/arm.armv7/usr/fbsd/stable-12-src/arm.armv7/tmp/legacy/usr/lib -L/usr/obj/rpi2_clang/arm.armv7/usr/fbsd/stable-12-src/arm.armv7/tmp/obj-tools/lib/libz -L/usr/obj/rpi2_clang/arm.armv7/usr/fbsd/stable-12-src/arm.armv7/tmp/obj-tools/lib/libexecinfo -L/usr/obj/rpi2_clang/arm.armv7/usr/fbsd/stable-12-src/arm.armv7/tmp/obj-tools/lib/libelf -L/usr/obj/rpi2_clang/arm.armv7/usr/fbsd/stable-12-src/arm.armv7/tmp/obj-tools/lib/ncurses/ncursesw -L/usr/obj/rpi2_clang/arm.armv7/usr/fbsd/stable-12-src/arm.armv7/tmp/obj-tools/lib/libthr -L/usr/lib --gc-sections cc1_main.o cc1as_main.o cc1gen_reproducer_main.o driver.o /usr/obj/rpi2_clang/arm.armv7/usr/fbsd/stable-12-src/arm.armv7/tmp/obj-tools/lib/clang/libclang/libclang.a /usr/obj/rpi2_clang/arm.armv7/usr/fbsd/stable-12-src/arm.armv7/tmp/obj-tools/lib/clang/libllvm/libllvm.a -lz -lexecinfo -lelf -lncursesw -lpthread -legacy -lc++ -lm -lgcc -lgcc_eh -lc -lgcc -lgcc_eh /usr/lib/crtend.o /usr/lib/crtn.o
#0 0x00de160c llvm::sys::PrintStackTrace(llvm::raw_ostream&) /usr/src/contrib/llvm-project/llvm/lib/Support/Unix/Signals.inc:572:7
#1 0x00ddf604 llvm::sys::RunSignalHandlers() /usr/src/contrib/llvm-project/llvm/lib/Support/Signals.cpp:0:5
#2 0x00de1f3c SignalHandler(int) /usr/src/contrib/llvm-project/llvm/lib/Support/Unix/Signals.inc:406:1
#3 0x42021db4 handle_signal /usr/src/lib/libthr/thread/thr_sig.c:0:3
ARM64TODO: fill_fpregs32c++: error: unable to execute command: Abort trap (core dumped)
c++: error: linker command failed due to signal (use -v to see invocation)
*** [clang.full] Error code 254
make[4]: stopped in /usr/fbsd/stable-12-src/usr.bin/clang/clang
.ERROR_TARGET='clang.full'
.ERROR_META_FILE='/usr/obj/rpi2_clang/arm.armv7/usr/fbsd/stable-12-src/arm.armv7/tmp/obj-tools/usr.bin/clang/clang/clang.full.meta'
.MAKE.LEVEL='4'
MAKEFILE=''
.MAKE.MODE='meta missing-filemon=yes missing-meta=yes silent=yes verbose'
_ERROR_CMD='c++ -O -pipe -fno-common -mlong-calls -I/usr/obj/rpi2_clang/arm.armv7/usr/fbsd/stable-12-src/arm.armv7/tmp/obj-tools/lib/clang/libclang -I/usr/obj/rpi2_clang/arm.armv7/usr/fbsd/stable-12-src/arm.armv7/tmp/obj-tools/lib/clang/libllvm -I/usr/fbsd/stable-12-src/contrib/llvm-project/clang/include -I/usr/fbsd/stable-12-src/lib/clang/include -I/usr/fbsd/stable-12-src/contrib/llvm-project/llvm/include -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -DHAVE_VCS_VERSION_INC -DNDEBUG -DLLVM_DEFAULT_TARGET_TRIPLE=\"armv7-unknown-freebsd12.2-gnueabihf\" -DLLVM_HOST_TRIPLE=\"armv7-unknown-freebsd12.2\" -DDEFAULT_SYSROOT=\"/usr/obj/rpi2_clang/arm.armv7/usr/fbsd/stable-12-src/arm.armv7/tmp\" -DLLVM_TARGET_ENABLE_ARM -DLLVM_NATIVE_ASMPARSER=LLVMInitializeARMAsmParser -DLLVM_NATIVE_ASMPRINTER=LLVMInitializeARMAsmPrinter -DLLVM_NATIVE_DISASSEMBLER=LLVMInitializeARMDisassembler -DLLVM_NATIVE_TARGET=LLVMInitializeARMTarget -DLLVM_NATIVE_TARGETINFO=LLVMInitializeARMTargetInfo -DLLVM_NATIVE_TARGETMC=LLVMInitializeARMTargetMC -ffunction-sections -fdata-sections -gline-tables-only -Wno-format-zero-length -mcpu=cortex-a7 -Qunused-arguments -I/usr/obj/rpi2_clang/arm.armv7/usr/fbsd/stable-12-src/arm.armv7/tmp/legacy/usr/include -fno-exceptions -fno-rtti -std=c++14 -mcpu=cortex-a7 -stdlib=libc++ -Wno-c++11-extensions -Wl,--gc-sections -static -L/usr/obj/rpi2_clang/arm.armv7/usr/fbsd/stable-12-src/arm.armv7/tmp/legacy/usr/lib -o clang.full cc1_main.o cc1as_main.o cc1gen_reproducer_main.o driver.o /usr/obj/rpi2_clang/arm.armv7/usr/fbsd/stable-12-src/arm.armv7/tmp/obj-tools/lib/clang/libclang/libclang.a /usr/obj/rpi2_clang/arm.armv7/usr/fbsd/stable-12-src/arm.armv7/tmp/obj-tools/lib/clang/libllvm/libllvm.a -L/usr/obj/rpi2_clang/arm.armv7/usr/fbsd/stable-12-src/arm.armv7/tmp/obj-tools/lib/libz -lz -L/usr/obj/rpi2_clang/arm.armv7/usr/fbsd/stable-12-src/arm.armv7/tmp/obj-tools/lib/libexecinfo -lexecinfo -L/usr/obj/rpi2_clang/arm.armv7/usr/fbsd/stable-12-src/arm.armv7/tmp/obj-tools/lib/libelf -lelf -L/usr/obj/rpi2_clang/arm.armv7/usr/fbsd/stable-12-src/arm.armv7/tmp/obj-tools/lib/ncurses/ncursesw -lncursesw -L/usr/obj/rpi2_clang/arm.armv7/usr/fbsd/stable-12-src/arm.armv7/tmp/obj-tools/lib/libthr -lpthread -legacy;'
.CURDIR='/usr/fbsd/stable-12-src/usr.bin/clang/clang'
.MAKE='make'
.OBJDIR='/usr/obj/rpi2_clang/arm.armv7/usr/fbsd/stable-12-src/arm.armv7/tmp/obj-tools/usr.bin/clang/clang'
.TARGETS='all'
DESTDIR=''
LD_LIBRARY_PATH=''
MACHINE='arm'
MACHINE_ARCH='armv7'
MAKEOBJDIRPREFIX=''
MAKESYSPATH='/usr/fbsd/stable-12-src/share/mk'
MAKE_VERSION='20201117'
PATH='/usr/obj/rpi2_clang/arm.armv7/usr/fbsd/stable-12-src/arm.armv7/tmp/legacy/usr/sbin:/usr/obj/rpi2_clang/arm.armv7/usr/fbsd/stable-12-src/arm.armv7/tmp/legacy/usr/bin:/usr/obj/rpi2_clang/arm.armv7/usr/fbsd/stable-12-src/arm.armv7/tmp/legacy/bin:/sbin:/bin:/usr/sbin:/usr/bin'
SRCTOP='/usr/fbsd/stable-12-src'
OBJTOP='/usr/obj/rpi2_clang/arm.armv7/usr/fbsd/stable-12-src/arm.armv7/tmp/obj-tools'
.MAKE.MAKEFILES='/usr/fbsd/stable-12-src/share/mk/sys.mk /usr/fbsd/stable-12-src/share/mk/local.sys.env.mk /usr/fbsd/stable-12-src/share/mk/src.sys.env.mk /root/src.configs/src.conf.armv7-clang-bootstrap.armv7-host /usr/fbsd/stable-12-src/share/mk/bsd.mkopt.mk /usr/fbsd/stable-12-src/share/mk/src.sys.obj.mk /usr/fbsd/stable-12-src/share/mk/auto.obj.mk /usr/fbsd/stable-12-src/share/mk/bsd.suffixes.mk /root/src.configs/make.conf /usr/fbsd/stable-12-src/share/mk/local.sys.mk /usr/fbsd/stable-12-src/share/mk/src.sys.mk /dev/null /usr/fbsd/stable-12-src/usr.bin/clang/clang/Makefile /usr/fbsd/stable-12-src/share/mk/src.opts.mk /usr/fbsd/stable-12-src/share/mk/bsd.own.mk /usr/fbsd/stable-12-src/share/mk/bsd.opts.mk /usr/fbsd/stable-12-src/share/mk/bsd.cpu.mk /usr/fbsd/stable-12-src/share/mk/bsd.compiler.mk /usr/fbsd/stable-12-src/share/mk/bsd.linker.mk /usr/fbsd/stable-12-src/usr.bin/clang/clang.prog.mk /usr/fbsd/stable-12-src/lib/clang/clang.pre.mk /usr/fbsd/stable-12-src/lib/clang/llvm.pre.mk /usr/fbsd/stable-12-src/lib/clang/clang.build.mk /usr/fbsd/stable-12-src/lib/clang/llvm.build.mk /usr/fbsd/stable-12-src/tools/build/mk/bsd.prog.mk /usr/fbsd/stable-12-src/share/mk/bsd.prog.mk /usr/fbsd/stable-12-src/share/mk/bsd.init.mk /usr/fbsd/stable-12-src/share/mk/local.init.mk /usr/fbsd/stable-12-src/share/mk/src.init.mk /usr/fbsd/stable-12-src/usr.bin/clang/clang/../Makefile.inc /usr/fbsd/stable-12-src/usr.bin/clang/clang/../../Makefile.inc /usr/fbsd/stable-12-src/share/mk/bsd.libnames.mk /usr/fbsd/stable-12-src/share/mk/src.libnames.mk /usr/fbsd/stable-12-src/share/mk/bsd.nls.mk /usr/fbsd/stable-12-src/share/mk/bsd.confs.mk /usr/fbsd/stable-12-src/share/mk/bsd.files.mk /usr/fbsd/stable-12-src/share/mk/bsd.dirs.mk /usr/fbsd/stable-12-src/share/mk/bsd.incs.mk /usr/fbsd/stable-12-src/share/mk/bsd.links.mk /usr/fbsd/stable-12-src/share/mk/bsd.dep.mk /usr/fbsd/stable-12-src/share/mk/bsd.clang-analyze.mk /usr/fbsd/stable-12-src/share/mk/bsd.obj.mk /usr/fbsd/stable-12-src/share/mk/bsd.subdir.mk /usr/fbsd/stable-12-src/share/mk/bsd.sys.mk /usr/fbsd/stable-12-src/tools/build/mk/Makefile.boot'
.PATH='. /usr/fbsd/stable-12-src/usr.bin/clang/clang /usr/fbsd/stable-12-src/contrib/llvm-project/clang/tools/driver'
1 error
# gdb /usr/bin/ld.lld /usr/obj/rpi2_clang/arm.armv7/usr/fbsd/stable-12-src/arm.armv7/tmp/obj-tools/usr.bin/clang/clang/ld.lld.core
GNU gdb (GDB) 10.1 [GDB v10.1 for FreeBSD]
Copyright (C) 2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
. . .
Reading symbols from /usr/bin/ld.lld...
Reading symbols from /usr/lib/debug//usr/bin/ld.lld.debug...
[New LWP 100183]
Core was generated by `/usr/bin/ld --eh-frame-hdr -Bstatic -o clang.full /usr/lib/crt1.o /usr/lib/crti.'.
Program terminated with signal SIGABRT, Aborted.
#0 thr_kill () at thr_kill.S:4
4 thr_kill.S: No such file or directory.
(gdb) info threads
Id Target Id Frame
* 1 LWP 100183 thr_kill () at thr_kill.S:4(gdb) info threads
Id Target Id Frame
* 1 LWP 100183 thr_kill () at thr_kill.S:4
(gdb) bt
#0 thr_kill () at thr_kill.S:4
#1 0x4227d998 in __raise (s=6) at /usr/src/lib/libc/gen/raise.c:52
#2 0x423322c8 in abort () at /usr/src/lib/libc/stdlib/abort.c:79
#3 0x00da5e4c in report_bad_alloc_error () at /usr/src/contrib/llvm-project/llvm/lib/Support/ErrorHandling.cpp:174
#4 0x00da61c8 in out_of_memory_new_handler() () at /usr/src/contrib/llvm-project/llvm/lib/Support/ErrorHandling.cpp:187
#5 0x420f5d24 in operator new (size=<optimized out>) at /usr/src/contrib/llvm-project/libcxx/src/new.cpp:73
#6 0x004d995c in Allocate () at /usr/src/contrib/llvm-project/llvm/include/llvm/Support/AllocatorBase.h:85
#7 StartNewSlab () at /usr/src/contrib/llvm-project/llvm/include/llvm/Support/Allocator.h:336
#8 Allocate () at /usr/src/contrib/llvm-project/llvm/include/llvm/Support/Allocator.h:188
#9 0x00543388 in Allocate () at /usr/src/contrib/llvm-project/llvm/include/llvm/Support/Allocator.h:202
#10 Allocate () at /usr/src/contrib/llvm-project/llvm/include/llvm/Support/AllocatorBase.h:46
#11 Allocate<lld::elf::Defined> () at /usr/src/contrib/llvm-project/llvm/include/llvm/Support/AllocatorBase.h:69
#12 Allocate () at /usr/src/contrib/llvm-project/llvm/include/llvm/Support/Allocator.h:427
#13 make<lld::elf::Defined, lld::elf::ObjFile<llvm::object::ELF32LE> *, lld::elf::StringRefZ &, llvm::ELF::(anonymous enum at /usr/src/contrib/llvm-project/llvm/include/llvm/BinaryFormat/ELF.h:1044:1), const unsigned char &, unsigned char &, const llvm::support::detail::packed_endian_specific_integral<unsigned int, llvm::support::little, 1, 1> &, const llvm::support::detail::packed_endian_specific_integral<unsigned int, llvm::support::little, 1, 1> &, lld::elf::InputSectionBase *&> () at /usr/src/contrib/llvm-project/lld/include/lld/Common/Memory.h:62
#14 initializeSymbols () at /usr/src/contrib/llvm-project/lld/ELF/InputFiles.cpp:1107
#15 0x0053a008 in parse () at /usr/src/contrib/llvm-project/lld/ELF/InputFiles.cpp:401
#16 doParseFile<llvm::object::ELFType<llvm::support::little, false> > () at /usr/src/contrib/llvm-project/lld/ELF/InputFiles.cpp:208
#17 parseFile () at /usr/src/contrib/llvm-project/lld/ELF/InputFiles.cpp:215
#18 0x0053b684 in fetch () at /usr/src/contrib/llvm-project/lld/ELF/InputFiles.cpp:1212
#19 0x005fdce8 in addSymbol () at /usr/src/contrib/llvm-project/lld/ELF/SymbolTable.cpp:98
#20 0x0053b4b4 in parse () at /usr/src/contrib/llvm-project/lld/ELF/InputFiles.cpp:1184
#21 0x005143bc in link<llvm::object::ELFType<llvm::support::little, false> > () at /usr/src/contrib/llvm-project/lld/ELF/Driver.cpp:1916
#22 0x0050c864 in main () at /usr/src/contrib/llvm-project/lld/ELF/Driver.cpp:539
#23 0x0050b078 in link () at /usr/src/contrib/llvm-project/lld/ELF/Driver.cpp:114
#24 0x006be290 in main () at /usr/src/contrib/llvm-project/lld/tools/lld/lld.cpp:146
The size of the slab being allocated and the allocation initiation
are from #7 above:
/// Allocate a new slab and move the bump pointers over into the new
/// slab, modifying CurPtr and End.
void StartNewSlab() {
size_t AllocatedSlabSize = computeSlabSize(Slabs.size());
void *NewSlab =
Allocator.Allocate(AllocatedSlabSize, alignof(std::max_align_t));
. . .
This code is associated with the class template:
/// Allocate memory in an ever growing pool, as if by bump-pointer.
///
/// This isn't strictly a bump-pointer allocator as it uses backing slabs of
/// memory rather than relying on a boundless contiguous heap. However, it has
/// bump-pointer semantics in that it is a monotonically growing pool of memory
/// where every allocation is found by merely allocating the next N bytes in
/// the slab, or the next N bytes in the next slab.
///
/// Note that this also has a threshold for forcing allocations above a certain
/// size into their own slab.
///
/// The BumpPtrAllocatorImpl template defaults to using a MallocAllocator
/// object, which wraps malloc, to allocate memory, but it can be changed to
/// use a custom allocator.
///
/// The GrowthDelay specifies after how many allocated slabs the allocator
/// increases the size of the slabs.
template <typename AllocatorT = MallocAllocator, size_t SlabSize = 4096,
size_t SizeThreshold = SlabSize, size_t GrowthDelay = 128>
class BumpPtrAllocatorImpl
: public AllocatorBase<BumpPtrAllocatorImpl<AllocatorT, SlabSize,
SizeThreshold, GrowthDelay>> {
. . .
The actual malloc call and the later out_of_memory_new_handler
call was via #5:
// Implement all new and delete operators as weak definitions
// in this shared library, so that they can be overridden by programs
// that define non-weak copies of the functions.
_LIBCPP_WEAK
void *
operator new(std::size_t size) _THROW_BAD_ALLOC
{
if (size == 0)
size = 1;
void* p;
while ((p = ::malloc(size)) == 0)
{
// If malloc fails and there is a new_handler,
// call it to try free up memory.
std::new_handler nh = std::get_new_handler();
if (nh)
nh();
. . .
malloc really did return NULL.
It looks like the allocations simply got to be to
big in total in process virtual memory space, not
necessarily in (contiguous) RAM space.
===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)
More information about the freebsd-arm
mailing list