Re: 14.0-CURRENT failed to reclaim memory error in RPi 3B build
Date: Wed, 02 Nov 2022 23:52:40 UTC
On 2022-Nov-2, at 14:09, Archimedes Gaviola <archimedes.gaviola@gmail.com> wrote: > On Mon, Oct 31, 2022 at 1:47 PM Archimedes Gaviola <archimedes.gaviola@gmail.com> wrote: > > . . . > > . . . > > > Hi Mark, > > Just an update, as kernel and world compilation is ongoing with my RPi3B system (with swap partition) is doing so far, so good. It already surpassed the tough part that breaks the compilation process here. > ... > > llvm-tblgen -gen-asm-matcher -I /usr/src/contrib/llvm-project/llvm/include -I /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV -d RISCVGenAsmMatcher.inc.d -o RISCVGenAsmMatcher.inc /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV/RISCV.td > llvm-tblgen -gen-asm-writer -I /usr/src/contrib/llvm-project/llvm/include -I /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV -d RISCVGenAsmWriter.inc.d -o RISCVGenAsmWriter.inc /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV/RISCV.td > llvm-tblgen -gen-callingconv -I /usr/src/contrib/llvm-project/llvm/include -I /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV -d RISCVGenCallingConv.inc.d -o RISCVGenCallingConv.inc /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV/RISCV.td > llvm-tblgen -gen-compress-inst-emitter -I /usr/src/contrib/llvm-project/llvm/include -I /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV -d RISCVGenCompressInstEmitter.inc.d -o RISCVGenCompressInstEmitter.inc /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV/RISCV.td > llvm-tblgen -gen-dag-isel -I /usr/src/contrib/llvm-project/llvm/include -I /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV -d RISCVGenDAGISel.inc.d -o RISCVGenDAGISel.inc /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV/RISCV.td > llvm-tblgen -gen-disassembler -I /usr/src/contrib/llvm-project/llvm/include -I /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV -d RISCVGenDisassemblerTables.inc.d -o RISCVGenDisassemblerTables.inc /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV/RISCV.td > llvm-tblgen -gen-global-isel -I /usr/src/contrib/llvm-project/llvm/include -I /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV -d RISCVGenGlobalISel.inc.d -o RISCVGenGlobalISel.inc /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV/RISCV.td > llvm-tblgen -gen-instr-info -I /usr/src/contrib/llvm-project/llvm/include -I /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV -d RISCVGenInstrInfo.inc.d -o RISCVGenInstrInfo.inc /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV/RISCV.td > llvm-tblgen -gen-emitter -I /usr/src/contrib/llvm-project/llvm/include -I /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV -d RISCVGenMCCodeEmitter.inc.d -o RISCVGenMCCodeEmitter.inc /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV/RISCV.td > llvm-tblgen -gen-pseudo-lowering -I /usr/src/contrib/llvm-project/llvm/include -I /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV -d RISCVGenMCPseudoLowering.inc.d -o RISCVGenMCPseudoLowering.inc /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV/RISCV.td > llvm-tblgen -gen-register-bank -I /usr/src/contrib/llvm-project/llvm/include -I /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV -d RISCVGenRegisterBank.inc.d -o RISCVGenRegisterBank.inc /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV/RISCV.td > llvm-tblgen -gen-register-info -I /usr/src/contrib/llvm-project/llvm/include -I /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV -d RISCVGenRegisterInfo.inc.d -o RISCVGenRegisterInfo.inc /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV/RISCV.td > llvm-tblgen -gen-searchable-tables -I /usr/src/contrib/llvm-project/llvm/include -I /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV -d RISCVGenSearchableTables.inc.d -o RISCVGenSearchableTables.inc /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV/RISCV.td > llvm-tblgen -gen-subtarget -I /usr/src/contrib/llvm-project/llvm/include -I /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV -d RISCVGenSubtargetInfo.inc.d -o RISCVGenSubtargetInfo.inc /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV/RISCV.td > llvm-tblgen -gen-searchable-tables -I /usr/src/contrib/llvm-project/llvm/include -I /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV -d RISCVGenSystemOperands.inc.d -o RISCVGenSystemOperands.inc /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV/RISCV.td > > Any thoughts why this part is quite a challenge when it comes to memory usage? The other architectures do not possess such behavior... just curious. I've not done any monitoring of buildworld buildkernel build activity (RAM use, memory space use, swap partition use over time) on RPi3B class hardware in a very long time. Even on systems that I have monitored in more recent times, what I usually monitor tends to be builds with -jN (such as -j4 fora 4-hardware-thread system). (I once did have an example of -j3 taking less time than -j4 on a RPi4B. Basically, the memory subsystem can be saturated without all the cores being in use. The extra interference made things take longer.) You had listed that you were using the likes of: # cd /usr/src ; make KERNCONF=ARM TARGET_ARCH=aarch64 \ buildkernel buildworld installkernel installworld distribution \ DESTDIR=/home/freebsd/rpi3b I'll note that the standard order of the first 2 is: buildworld buildkernel This is because buildworld builds some software that buildkernel does not build for itself but does use. There is a kernel-toolchain target for avoiding the need to do a full buildworld just to buildkernel , so: kernel-toolchain buildkernel is an expected sequence. I do not know how long a from-scratch buildworld buildkernel without a -jN takes on a RPi3B these days. If I remember right, for -jN with 1<N, I last saw claims about such they were somewhere in the range 36hr..48hr. But I'm unsure of the specific N that was in use. Nor do I know the storage media type(s) involved, for example. I do not remember any reports for -j1 . Use of the likes of: vm.pageout_oom_seq=120 was essential to such -jN usage on a RPi3B as N gets larger. Of course, swap partition use for paging was also essential. Likewise use of: vm.swap_enabled=0 vm.swap_idle_enabled=0 can be important to not losing communication with the RPi3B. Those last 2 are not tunable but are writable: # sysctl -aT | grep swap_ # sysctl -aW | grep swap_ vm.swap_enabled: 0 . . . vm.swap_idle_enabled: 0 . . . (This means that they have fewer places where assignments can be made. For example, the loader can not make the assignments.) By contrast, vm.pageout_oom_seq is both writable and tunable: # sysctl -aW | grep oom . . . vm.pageout_oom_seq: 120 . . . # sysctl -aT | grep oom . . . vm.pageout_oom_seq: 120 . . . (So even the loader can make such assignments.) I'll note that I've no interest in using arm hardware to build for other types of hardware. So I normally have the targeting of support for building for other architectures disabled when I build on aarch64 or armv7. (Basically, a less complete clang/clang++ related toolchain ends up being built.) === Mark Millard marklmi at yahoo.com