From nobody Tue Sep 03 21:47:06 2024 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Wyzjl31jJz5MTj0; Tue, 03 Sep 2024 21:47:07 +0000 (UTC) (envelope-from brooks@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R10" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Wyzjl2PxWz43vZ; Tue, 3 Sep 2024 21:47:07 +0000 (UTC) (envelope-from brooks@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1725400027; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=hyPJ40xTOzUcguhYJn+1V1WFij52cPc0WTRyVArN250=; b=sI8HP2uZsIHwVYcs1PJDZTnpmndu7z5AaeKneicEg+xXFE9uVYEIt/an0s9/OnPn79y8og 0rKWZAie2a3YX5X47PrrG5g+sR824ZkUoTNcT/T0AIzZDwo7pdeg5xPPKj3HMX9wRNtd2P pDPfb8EwvZEhCGpc9NATKLdy5JhzWogPBXoD/0cKBuYcbdhnMedKsVlPmLBPLwx5qbN4hW AvKLqfxRchwkUXhvIY+UKYjrbTf83sMkGl98MtgNWpA4vAQ2oZKgh5QSOU1YXlrGDL6p67 Q5zEUDmh+kH+B3gQ5ZGrZuNJ4mQmnn8UL5GknvMl0DdJ0mQneYiqjBRYZsku/w== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1725400027; a=rsa-sha256; cv=none; b=YiAxDXH4ODPxWi3sIBwIqp1qB25EVj5eLyRMb159sNyE3EyDulqhcpZd20tlbb8adh8/D1 FT+DUGti7loLPiuw+c/8AmE8McG/XndSXidhaHWaUmVNYAzZW1QuvJ2bOs4lIe7jMXlICP OVo7aUHCY+EsqCdDGkC9FygqX9E65nGnear5+OpgzE/N+bDhHownhxjTwIwakMoNFuvDLs bu8bEf73sRhzRARC0NL5HXQZ2o81L3JADUvJuOtdDnhivHb7qZ6boDu0hmZl8FV2BjfM6n DW1EGn4LaS6mPMJzEdZbRsHy6gMzy60Qte2yUalBWT9o0Ub4JOuhvqdlA7giZA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1725400027; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=hyPJ40xTOzUcguhYJn+1V1WFij52cPc0WTRyVArN250=; b=LZi82RKpU7VjY1bfdyzKb4GhFhc7gurmeU6ws3uC9q4JY7782e4vSoUr3Trpp9dJMLoPEo mygnHy6oJ/6KM6LY9LT+h6t2u8bqsYvjijzhZCkp8fzAQkemIdHwPbEDtjXe8Hls8Hubti X4/uLJyi4gAuLZ7q3LEZ7ZxGaw3nkZ2UtWkAVtJ2uXKVE+TKO58Awh2lXOCVV+0C4Nib7J FUfhNN8BEN9ujOf3Ezw992Bnxfo3uiQaDbFxlMokeJr83UAQ/gh/noI8j3HukT4CbmNOJg S8v/y9HO7EvCGGcSOtO9YcxGxE6emf8qh4OlnLXjK0qTZEp3BHCuN71YrPqWqg== Received: from spindle.one-eyed-alien.net (spindle.one-eyed-alien.net [199.48.129.229]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: brooks/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 4Wyzjl1KJqz1GsY; Tue, 3 Sep 2024 21:47:07 +0000 (UTC) (envelope-from brooks@freebsd.org) Received: by spindle.one-eyed-alien.net (Postfix, from userid 3001) id CE0043C019B; Tue, 03 Sep 2024 21:47:06 +0000 (UTC) Date: Tue, 3 Sep 2024 21:47:06 +0000 From: Brooks Davis To: Mark Millard Cc: FreeBSD Mailing List , FreeBSD ARM List , FreeBSD Toolchain , Tomoaki AOKI Subject: Re: For an armv7 context, /usr/local/llvm1[789]/lib/clang/1[789]/include/arm_bf16.h does not exist: one thing blocking a firefox build via llvm1[78] Message-ID: References: <75609A57-7B50-40F5-88A8-0278CCCC018B@yahoo.com> <5BEBBFCB-A877-4124-B07F-D4C6D25B7026@yahoo.com> <02BE22BE-7E86-43D1-86D8-85A527625AD6@yahoo.com> List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <02BE22BE-7E86-43D1-86D8-85A527625AD6@yahoo.com> On Tue, Sep 03, 2024 at 02:36:19PM -0700, Mark Millard wrote: > On Sep 3, 2024, at 13:32, Brooks Davis wrote: > > > On Tue, Sep 03, 2024 at 12:09:55PM -0700, Mark Millard wrote: > >> On Sep 3, 2024, at 11:12, Brooks Davis wrote: > >> > >>> On Fri, Aug 30, 2024 at 10:05:06PM -0700, Mark Millard wrote: > >>>> On Aug 30, 2024, at 21:26, Mark Millard wrote: > >>>> > >>>>> On Aug 30, 2024, at 20:33, Mark Millard wrote: > >>>>> > >>>>>> [Subject was retitled.] > >>>>>> > >>>>>> On Aug 30, 2024, at 16:24, Mark Millard wrote: > >>>>>> > >>>>>>> What my test-of-building got was: No include file found and > >>>>>>> no OFlags::TMPFILE found (OFlags:: was found, TMPFILE in OFlags:: was not): > >>>>>>> > >>>>>>> In file included from /wrkdirs/usr/ports/www/firefox/work/firefox-129.0.2/mfbt/lz4/xxhash.c:43: > >>>>>>> In file included from /wrkdirs/usr/ports/www/firefox/work/firefox-129.0.2/mfbt/lz4/xxhash.h:3434: > >>>>>>> /usr/local/llvm17/lib/clang/17/include/arm_neon.h:37:10: fatal error: 'arm_bf16.h' file not found > >>>>>>> 37 | #include > >>>>>>> | ^~~~~~~~~~~~ > >>>>>>> . . . > >>>>>>> > >>>>>>> error[E0599]: no associated item named `TMPFILE` found for struct `backend::fs::types::OFlags` in the current scope > >>>>>>> --> /wrkdirs/usr/ports/www/firefox/work/firefox-129.0.2/third_party/rust/rustix/src/backend/libc/fs/syscalls.rs:144:32 > >>>>>>> | > >>>>>>> 144 | if oflags.contains(OFlags::TMPFILE) && crate::backend::if_glibc_is_less_than_2_25() { > >>>>>>> | ^^^^^^^ associated item not found in `OFlags` > >>>>>>> | > >>>>>>> ::: /wrkdirs/usr/ports/www/firefox/work/firefox-129.0.2/third_party/rust/rustix/src/backend/libc/fs/types.rs:203:1 > >>>>>>> | > >>>>>>> 203 | / bitflags! { > >>>>>>> 204 | | /// `O_*` constants for use with [`openat`]. > >>>>>>> 205 | | /// > >>>>>>> 206 | | /// [`openat`]: crate::fs::openat > >>>>>>> ... | > >>>>>>> 333 | | } > >>>>>>> 334 | | } > >>>>>>> | |_- associated item `TMPFILE` not found for this struct > >>>>>>> | > >>>>>>> . . . > >>>>>>> = note: this error originates in the macro `$crate::__impl_bitflags` which comes from the expansion of the macro `bitflags` (in Nightly builds, run with -Z macro-backtrace for more info) > >>>>>>> > >>>>>>> . . . > >>>>>>> > >>>>>>> error[E0599]: no associated item named `TMPFILE` found for struct `backend::fs::types::OFlags` in the current scope > >>>>>>> --> /wrkdirs/usr/ports/www/firefox/work/firefox-129.0.2/third_party/rust/rustix/src/backend/libc/fs/syscalls.rs:207:32 > >>>>>>> | > >>>>>>> 207 | if oflags.contains(OFlags::TMPFILE) && crate::backend::if_glibc_is_less_than_2_25() { > >>>>>>> | ^^^^^^^ associated item not found in `OFlags` > >>>>>>> | > >>>>>>> ::: /wrkdirs/usr/ports/www/firefox/work/firefox-129.0.2/third_party/rust/rustix/src/backend/libc/fs/types.rs:203:1 > >>>>>>> | > >>>>>>> 203 | / bitflags! { > >>>>>>> 204 | | /// `O_*` constants for use with [`openat`]. > >>>>>>> 205 | | /// > >>>>>>> 206 | | /// [`openat`]: crate::fs::openat > >>>>>>> ... | > >>>>>>> 333 | | } > >>>>>>> 334 | | } > >>>>>>> | |_- associated item `TMPFILE` not found for this struct > >>>>>>> | > >>>>>>> . . . > >>>>>>> = note: this error originates in the macro `$crate::__impl_bitflags` which comes from the expansion of the macro `bitflags` (in Nightly builds, run with -Z macro-backtrace for more info) > >>>>>>> > >>>>>>> . . . > >>>>>>> = note: this error originates in the macro `$crate::__impl_bitflags` which comes from the expansion of the macro `bitflags` (in Nightly builds, run with -Z macro-backtrace for more info) > >>>>>>> > >>>>>>> For more information about this error, try `rustc --explain E0599`. > >>>>>>> error: could not compile `rustix` (lib) due to 2 previous errors > >>>>>>> > >>>>>>> > >>>>>>> For reference: > >>>>>>> > >>>>>>> # uname -apKU > >>>>>>> FreeBSD aarch64-main-pbase 15.0-CURRENT FreeBSD 15.0-CURRENT #8 main-n271819-5cbb98c8259c-dirty: Fri Aug 23 22:06:47 PDT 2024 root@aarch64-main-pbase:/usr/obj/BUILDs/main-CA76-nodbg-clang/usr/main-src/arm64.aarch64/sys/GENERIC-NODBG-CA76 arm64 aarch64 1500023 1500023 > >>>>>>> > >>>>>>> # ~/fbsd-based-on-what-commit.sh -C /usr/ports/ > >>>>>>> 87a38a839ab8 (HEAD -> main, freebsd/main, freebsd/HEAD) net-im/dissent: update package description > >>>>>>> Author: Jan Beich > >>>>>>> Commit: Jan Beich > >>>>>>> CommitDate: 2024-08-24 18:30:01 +0000 > >>>>>>> branch: main > >>>>>>> merge-base: 87a38a839ab83c2def100a0975a7afb29e873cf2 > >>>>>>> merge-base: CommitDate: 2024-08-24 18:30:01 +0000 > >>>>>>> n674987 (--first-parent --count for merge-base) > >>>>>>> > >>>>>>> But firefox was updated to use: nss>=3.103:security/nss to match what was > >>>>>>> available. > >>>>>> > >>>>>> > >>>>>> Using devel/llvm18 instead got the same. > >>>>>> > >>>>>> Looking inside even a /usr/local/llvm19/lib/clang/19/include/ > >>>>>> also shows the arm_bf16.h file is not present. By contrast, > >>>>>> for an aarch64 context: > >>>>>> > >>>>>> # file /usr/local/llvm19/lib/clang/19/include/arm_bf16.h > >>>>>> /usr/local/llvm19/lib/clang/19/include/arm_bf16.h: C source, ASCII text > >>>>>> > >>>>>> Looking quickly at more llvm* shows: > >>>>>> > >>>>>> # grep -r arm_bf16 /usr/ports/devel/llvm1*/ | more > >>>>>> /usr/ports/devel/llvm11/pkg-plist:%%CLANG%%llvm%%LLVM_SUFFIX%%/lib/clang/%%LLVM_RELEASE%%/include/arm_bf16.h > >>>>>> /usr/ports/devel/llvm12/pkg-plist:%%CLANG%%llvm%%LLVM_SUFFIX%%/lib/clang/%%LLVM_RELEASE%%/include/arm_bf16.h > >>>>>> /usr/ports/devel/llvm13/pkg-plist:%%CLANG%%llvm%%LLVM_SUFFIX%%/lib/clang/%%LLVM_RELEASE%%/include/arm_bf16.h > >>>>>> /usr/ports/devel/llvm14/Makefile:_BE_INCS_ARM= arm_bf16.h arm_cde.h arm_fp16.h arm_mve.h arm_neon.h arm_sve.h > >>>>>> /usr/ports/devel/llvm15/Makefile:_BE_INCS_ARM= arm_bf16.h arm_cde.h arm_fp16.h arm_mve.h arm_neon.h arm_sve.h > >>>>>> /usr/ports/devel/llvm16/files/patch-backport-llvm-db49231: `arm_sve.h` and `arm_bf16.h`, and all those generated files will contain a > >>>>>> /usr/ports/devel/llvm16/files/patch-backport-llvm-db49231: `arm_bf16.h` immediately before their own typedef: > >>>>>> /usr/ports/devel/llvm16/files/patch-backport-llvm-db49231: #include > >>>>>> /usr/ports/devel/llvm16/files/patch-backport-llvm-db49231: Since `arm_bf16.h` is very likely supposed to be the one true place where > >>>>>> /usr/ports/devel/llvm16/files/patch-backport-llvm-db49231: OS << "#include \n"; > >>>>>> /usr/ports/devel/llvm16/files/patch-backport-llvm-db49231: OS << "#include \n"; > >>>>>> /usr/ports/devel/llvm16/Makefile:_BE_INCS_ARM= arm_bf16.h arm_cde.h arm_fp16.h arm_mve.h arm_neon.h arm_sve.h > >>>>>> /usr/ports/devel/llvm17/Makefile:_BE_INCS_AArch64= arm_bf16.h arm_sme_draft_spec_subject_to_change.h > >>>>>> /usr/ports/devel/llvm18/Makefile:_BE_INCS_AArch64= arm_bf16.h > >>>>>> /usr/ports/devel/llvm19/Makefile:_BE_INCS_AArch64= arm_bf16.h > >>>>>> > >>>>>> llvm1[456] had _BE_INCS_ARM containing arm_bf16.h (and more). > >>>>>> llvm1[789] do not. > >>>>>> > >>>>>> I wonder if: > >>>>>> > >>>>>> https://cgit.freebsd.org/ports/commit/devel/llvm17/Makefile?id=778e212f234a825c5e19612df4be2e8f838cb024 > >>>>>> > >>>>>> doing: > >>>>>> > >>>>>> -_BE_INCS_ARM= arm_bf16.h arm_cde.h arm_fp16.h arm_mve.h arm_neon.h arm_sve.h > >>>>>> +_BE_INCS_ARM= arm_cde.h arm_fp16.h arm_mve.h arm_neon.h arm_sve.h > >>>>>> > >>>>>> was correct. I'll note that in an armv7 context: > >>>>>> > >>>>>> # find /usr/local/*/gcc14/ -name arm_bf16.h -print > >>>>>> /usr/local/lib/gcc14/gcc/armv7-portbld-freebsd15.0/14.2.0/include/arm_bf16.h > >>>>>> > >>>>>> suggesting that gcc14 considers the file as not aarch64 specific but > >>>>>> as armv7 compatibile. > >>>>> > >>>>> I got that wrong! arm vs. aarch64 have different source files with the > >>>>> same name (under different paths): > >>>>> > >>>>> gcc/gcc/config/arm/arm_bf16.h has guard test: #ifndef _GCC_ARM_BF16_H > >>>>> gcc/gcc/config/aarch64/arm_bf16.h has guard test: #ifndef _AARCH64_BF16_H_ > >>>>> > >>>>> (More content is different.) > >>>> > >>>> As for llvm*: > >>>> > >>>> clang/lib/Basic/Targets/ARM.cpp has: > >>>> > >>>> if (HasBFloat16) { > >>>> Builder.defineMacro("__ARM_FEATURE_BF16", "1"); > >>>> Builder.defineMacro("__ARM_FEATURE_BF16_VECTOR_ARITHMETIC", "1"); > >>>> Builder.defineMacro("__ARM_BF16_FORMAT_ALTERNATIVE", "1"); > >>>> } > >>>> > >>>> clang/lib/Basic/Targets/AArch64.cpp has: > >>>> > >>>> if (HasBFloat16) { > >>>> Builder.defineMacro("__ARM_FEATURE_BF16", "1"); > >>>> Builder.defineMacro("__ARM_FEATURE_BF16_VECTOR_ARITHMETIC", "1"); > >>>> Builder.defineMacro("__ARM_BF16_FORMAT_ALTERNATIVE", "1"); > >>>> } > >>>> > >>>> which suggests bf16 support has 32-bit support (even if it is armv8 > >>>> 32-bit). Looking for AArch32 state in: > >>>> > >>>> DDI0487K_a_a-profile_architecture_reference_manual.pdf > >>>> > >>>> it says (via the AArch32 column of a table): > >>>> > >>>> BF16 Supported if FEAT_AA32BF16 is implemented. > >>>> > >>>> Looks to me like the removal of arm_bf16.h for llvm target ARM > >>>> was incorrect. > >>> > >>> The commit to the port simply refects changes upstream which made > >>> arm_br16.h aarch64-only. It was done in a massive commit (a70cf56d20b956) > >>> so may well have been wrong and no one notices because they always build > >>> all the backends. > >> > >> Hmm. > >> > >> My build in a armv7 context with arm_bf16.h listed was of: > >> > >> ---Begin OPTIONS List--- > >> ===> The following configuration options are available for llvm19-19.1.0.r3: > >> BE_AMDGPU=off: AMD GPU backend (required by mesa) > >> BE_WASM=off: WebAssembly backend (required by firefox via wasi) > >> CLANG=on: Build clang > >> DOCS=on: Build and/or install documentation > >> EXTRAS=on: Extra clang tools > >> LIT=on: Install lit and FileCheck test tools > >> LLD=on: Install lld, the LLVM linker > >> LLDB=on: Install lldb, the LLVM debugger > >> PYCLANG=on: Install python bindings to libclang > >> STATIC_LIBS=on: Install static libraries (does not effect sanitizers) > >> ====> Options available for the single BACKENDS: you have to select exactly one of them > >> BE_FREEBSD=off: Backends for FreeBSD architectures > >> BE_NATIVE=on: Backend(s) for this architecture (ARM) > >> BE_STANDARD=off: All non-experimental backends > >> ===> Use 'make config' to modify these settings > >> ---End OPTIONS List--- > >> > >> Note the ARM (no AArch64). > >> > >> It built just fine. When I looked at how arm_bf16.h is provided, > >> it looked to be built via llvm tools and is not direct source > >> code that as been committed. It looked to me like arm_bf16.h for > >> an ARM build got ARM (32-bit) content. (But I'm not expert.) > >> > >> The firefox build attempt found the file and got no complaints > >> about using the file. Sure looked to me like it just worked > >> when it is also listed in _BE_INCS_ARM . > > > > It's weird that it's getting installed at all. I'm rather confused. Could > > you do a `make check-plist` and send me the output? > > The below is about my personal builds in this time frame, > the ones that found and used the arm_b16.h file in a armv7 > context . . . > > Reminder/notice of context: I was experimenting with trying to > build firefox to see if BE_WASM being enabled was remotely worth > it for now. (It is not: firefox is no where near buildable for > armv7 at this point. BE_WASM just wastes cycles and such.) > > firefox uses '#include ' for armv7 builds and gets an > error about it being missing if it is not installed by the llvm > that it is using. So I had experimented with use of reverting the > _BE_INCS_ARM related commit material at issue, resulting in: > > -_BE_INCS_ARM= arm_cde.h arm_fp16.h arm_mve.h arm_neon.h arm_sme.h \ > +_BE_INCS_ARM= arm_bf16.h arm_cde.h arm_fp16.h arm_mve.h arm_neon.h arm_sme.h \ > > This was in order to see if such would allow firefox to get past > the specific error. It did get past that error point without any > complaints. > > Does that addition of arm_bf16.h to _BE_INCS_ARM remove the > confusion? > > Attempting to use 'make check-plist' started trying to build and > install a bunch of stuff so I stopped it. I do not plan on > rebuilding the llvm18 prerequisites via make. I've uninstalled > and cleaned out what installed before I stopped it. (Nearly all > my build activity is via poudriere-devel instead of the likes of > use of make or portmaster.) I've gone ahead and added arm_bf16.h back in my latest llvm19 update. (I just realized I did something more convoluted than necessicary and will fix it in the next update.) I'm still not sure how a file in a list in cmake named aarch64_only_generated_files ends up on an ARM only build, but so it goes. I'll merge to 17 and 18 as time permits. -- Brooks