From nobody Mon Mar 20 21:24:36 2023 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PgSS04kyJz40HZB for ; Mon, 20 Mar 2023 21:24:48 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-ed1-x533.google.com (mail-ed1-x533.google.com [IPv6:2a00:1450:4864:20::533]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4PgSS033SYz3rm7 for ; Mon, 20 Mar 2023 21:24:48 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-ed1-x533.google.com with SMTP id h8so52236530ede.8 for ; Mon, 20 Mar 2023 14:24:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20210112.gappssmtp.com; s=20210112; t=1679347487; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=SK0zLKhGtwhqFMaeuPvJUXD+dzCek3DSh1/qWtx13hg=; b=flaLeRNBYGgJO3KSeVXWjYDtRGSKmQc/XvtH/PR5Y5UevTelhx0mmIB/8jOANSs8i2 w8pyhjPwA5mIGTTPwSMy/0Ep708wuHalSYgM0gZ7Alg7ef+m/7L0tfM5BUx+K/stMKa+ ThgdSHAIymxZQPGaB7itql/Z6gaH1z1lW4gIjV7piLhVzTIYC9JjL0Zpu9Fu0L7OP9ow gyw3xU0F+EIr69h29Dln4L/g60WixOYxVTo18rt+i5uZNlEppqSKUpAHjc0htW22dC0o tNHD9EYWn027SKCUNPFwNjIaqP1pPRqSMfVdOeayVRMY8CPlZYh7QmaEGKqpSczFJ7Tq UB/g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679347487; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=SK0zLKhGtwhqFMaeuPvJUXD+dzCek3DSh1/qWtx13hg=; b=Kf+r36KRmj13jyY9T31aKGk/8wkbZLmZVEBavf4vrFSik6uEuMHPVlMT5y8AqX35LS KhLuuaRHzf/5XKKZVQ3nOUlWJJmio/51je7CYBjZGsf5t/Aebqtu5m3elncEir7aWONy J3+6B7PnKd/8QkdQDIjc+O/JfBfra25jfiuWqOmI8ftvWg4ubuMSTBcsHlAYLfg944op QJ5mMkJc1eGIaINja8kxJKFKVrXKVlY6qo2BKkpGrFTGVPLKay27EK7qRyZsmdyEoLgC 6QbXgAQC+j2MtX3ZrQxE+jpRM6hjn/foRF5Xq8Bv62WqErho+nspbJt0I7zBSUGO2pRx 9Djg== X-Gm-Message-State: AO0yUKUyyUP1zYuFZwl4U9FEX7yQkE3tWimB6bzYrYywQ8EF2HY4hfjH wJqet+G15PWi9zAEJNiv2Cb/KvS7vIg4mRKMoSK5Kw== X-Google-Smtp-Source: AK7set/KFA1Z/TIeUaSjs/O3F4ALYtzeL3fPsvEM/9h7r6OL47RBkf8ii40V8Y6LZ5AuSk6pVICfSNOGnwuvjZ0jL8U= X-Received: by 2002:a17:907:7d87:b0:931:f8b1:4474 with SMTP id oz7-20020a1709077d8700b00931f8b14474mr251589ejc.2.1679347487210; Mon, 20 Mar 2023 14:24:47 -0700 (PDT) List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org MIME-Version: 1.0 References: <20230320202847.GW2347@FreeBSD.org> <477846FC-FC56-4A2E-B2BD-FB98500B0F7F@yahoo.com> <20230320210353.GZ2347@FreeBSD.org> In-Reply-To: <20230320210353.GZ2347@FreeBSD.org> From: Warner Losh Date: Mon, 20 Mar 2023 15:24:36 -0600 Message-ID: Subject: Re: Poudriere friendly armv7 relases To: Glen Barber Cc: Mark Millard , "freebsd-arm@freebsd.org" Content-Type: multipart/alternative; boundary="0000000000006e129005f75b8efb" X-Rspamd-Queue-Id: 4PgSS033SYz3rm7 X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N --0000000000006e129005f75b8efb Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Mon, Mar 20, 2023 at 3:03=E2=80=AFPM Glen Barber wrote= : > On Mon, Mar 20, 2023 at 01:51:27PM -0700, Mark Millard wrote: > > On Mar 20, 2023, at 13:28, Glen Barber wrote: > > > > > On Mon, Mar 20, 2023 at 02:06:50PM -0600, Warner Losh wrote: > > >> Greetings, > > >> > > >> Since it looks like we're going to retain at least armv7 for FreeBSD > 14 > > >> (armv6 has been nominated for deprecation, but if it isn't > deprecated, all > > >> this applies to it). > > >> > > >> I'd like to start making at least the base.tgz, etc available for > armv7. > > >> This would allow us to create armv7 poduriere jails without building > from > > >> source. > > >> > > >> Is there some reason we're not doing this today? I know ISOs don't > make a > > >> lot of sense in the arm ecosystem, but having these artifacts would > enable > > >> poudriere binary install support. > > >> > > >> Comments? > > >> > > > > > > Several. :) > > > > > > I have looked into this in the past, and mhorne@ had even added some > > > environment knobs to the way armv7 is built, however I later realized > > > that it was not 1:1 compatible with how base.txz, etc., are generated > > > for other architectures. > > > > > > 1) For other architectures, base.txz is result of the 'ftp' target in > > > /usr/src/release. > > > > > > 2) armv7 does not have an 'ftp' target. (Well, it does not *disallow= * > > > it, and probably should at the immediate moment, but it does blow > > > up.) > > > > > > 3) Most importantly, and the reason I stopped looking further into > this, > > > we cannot run native armv7 binaries on an amd64 system (at least, > > > last I was aware). > > > > Does chroot and the like count for your purpose? > > > > armv7 packages are built without qemu or the like's > > involvement: > > > > default 131releng-armv7 on ampere3 > > quarterly 131releng-armv7 on ampere1 > > default main-armv7 on ampere2 > > > > This has been going on since 2022-Aug or so. > > > > These are natively built on arm64 hardware. > I should have pointed out that my requested change is useful for this as well. Independent of my desire to have it for cross builds, is the need for this setup to be installed w/o building from source. Warner > > I personally build for armv7 on a HoneyComb > > and have done so on a RPi4B in the past. (This > > is both system builds and package builds.) > > > > Basically all these machines support AArch32 > > in addition to AArch64: > > > > # sysctl kern.supported_archs > > kern.supported_archs: aarch64 armv7 > > > > > > > Particularly, we can only actually use what is > > > listed in kern.supported_archs, > > > > The ampere*'s should list armv7 in addition to aach64. > > (I've no access of my own to directly validate but > > given that ports are turned into packages . . .) > > > > aarch64 and armv7 are indeed listed. > > > > at least without falling back to some > > > sort of emulation or wrapper support (such as qemu or the like). > > > > Should not be needed, presuming access to have > > jobs run on one or more ampere* systems. > > > > The release build machines are (by design) kept separate from the rest > of the infrastructure within which we operate. (Same for the package > builders, as well.) > > > > Back when armv6 and armv7 support was added using shell scripts inste= ad > > > of hooking into release/Makefile, having a base.txz did not make much > > > sense because there were different environment variables that were > > > passed into the resulting output, some of which affected the loader > > > output, etc., specifically with regard to u-boot. I am not sure if > this > > > is still an issue or a concern, however. > > > > QUOTE > > author Emmanuel Vadot 2021-05-11 18:27:14 +0000 > > committer Emmanuel Vadot 2021-05-11 20:22:54 +0000 > > commit 0d6e5081eb0080c4703f1c5cc69c34f38d9149b7 (patch) > > tree a22f954f3003c1361f4ea5a411e92759a80c9089 /sysutils/u-boot-master > > parent c5fd1c2e186abb2e3209fa48d75d8dcdcda63f06 (diff) > > download ports-0d6e5081eb0080c4703f1c5cc69c34f38d9149b7.tar.gz > > ports-0d6e5081eb0080c4703f1c5cc69c34f38d9149b7.zip > > > > sysutils/u-boot-*: Remove ubldr support > > > > We have been using loader.efi on armv7 for a long time now. Remove > support for booting with ubldr and the needed patches that were never > upstreamed. While here add CONFIG_EFI_GRUB_ARM32_WORKAROUND=3Dy in the > Fragment as it's needed to have the cache flushed for us when loader.efi = is > started. > > END QUOTE > > > > > > So: before 2021-Jun. > > > > Noted. Thank you for looking. > > > > That said, I can take a look and see if we can package base.txz for > > > armv7, however I would like to do some archaeology work here to be su= re > > > that the resultant output is not going to have unexpected behavior > > > because of the userland not matching 100% the target SoC. > > > > > > Glen > > --0000000000006e129005f75b8efb Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


=
On Mon, Mar 20, 2023 at 3:03=E2=80=AF= PM Glen Barber <gjb@freebsd.org&g= t; wrote:
On Mon= , Mar 20, 2023 at 01:51:27PM -0700, Mark Millard wrote:
> On Mar 20, 2023, at 13:28, Glen Barber <gjb@freebsd.org> wrote:
>
> > On Mon, Mar 20, 2023 at 02:06:50PM -0600, Warner Losh wrote:
> >> Greetings,
> >>
> >> Since it looks like we're going to retain at least armv7 = for FreeBSD 14
> >> (armv6 has been nominated for deprecation, but if it isn'= t deprecated, all
> >> this applies to it).
> >>
> >> I'd like to start making at least the base.tgz, etc avail= able for armv7.
> >> This would allow us to create armv7 poduriere jails without b= uilding from
> >> source.
> >>
> >> Is there some reason we're not doing this today? I know I= SOs don't make a
> >> lot of sense in the arm ecosystem, but having these artifacts= would enable
> >> poudriere binary install support.
> >>
> >> Comments?
> >>
> >
> > Several.=C2=A0 :)
> >
> > I have looked into this in the past, and mhorne@ had even added s= ome
> > environment knobs to the way armv7 is built, however I later real= ized
> > that it was not 1:1 compatible with how base.txz, etc., are gener= ated
> > for other architectures.
> >
> > 1) For other architectures, base.txz is result of the 'ftp= 9; target in
> >=C2=A0 =C2=A0/usr/src/release.
> >
> > 2) armv7 does not have an 'ftp' target.=C2=A0 (Well, it d= oes not *disallow*
> >=C2=A0 =C2=A0it, and probably should at the immediate moment, but = it does blow
> >=C2=A0 =C2=A0up.)
> >
> > 3) Most importantly, and the reason I stopped looking further int= o this,
> >=C2=A0 =C2=A0we cannot run native armv7 binaries on an amd64 syste= m (at least,
> >=C2=A0 =C2=A0last I was aware).
>
> Does chroot and the like count for your purpose?
>
> armv7 packages are built without qemu or the like's
> involvement:
>
> default=C2=A0 =C2=A0131releng-armv7 on ampere3
> quarterly 131releng-armv7 on ampere1
> default=C2=A0 =C2=A0main-armv7=C2=A0 =C2=A0 =C2=A0 on ampere2
>
> This has been going on since 2022-Aug or so.
>

These are natively built on arm64 hardware.

=
I should have pointed out that my requested change is useful for this = as well.
Independent of my desire to have it for cross builds, is= the need for this setup
to be installed w/o building from source= .

Warner
=C2=A0
> I personally build for armv7 on a HoneyComb
> and have done so on a RPi4B in the past. (This
> is both system builds and package builds.)
>
> Basically all these machines support AArch32
> in addition to AArch64:
>
> # sysctl kern.supported_archs
> kern.supported_archs: aarch64 armv7
>
>
> > Particularly, we can only actually use what is
> >=C2=A0 =C2=A0listed in kern.supported_archs,
>
> The ampere*'s should list armv7 in addition to aach64.
> (I've no access of my own to directly validate but
> given that ports are turned into packages . . .)
>

aarch64 and armv7 are indeed listed.

> > at least without falling back to some
> >=C2=A0 =C2=A0sort of emulation or wrapper support (such as qemu or= the like).
>
> Should=C2=A0 not be needed, presuming access to have
> jobs run on one or more ampere* systems.
>

The release build machines are (by design) kept separate from the rest
of the infrastructure within which we operate.=C2=A0 (Same for the package<= br> builders, as well.)

> > Back when armv6 and armv7 support was added using shell scripts i= nstead
> > of hooking into release/Makefile, having a base.txz did not make = much
> > sense because there were different environment variables that wer= e
> > passed into the resulting output, some of which affected the load= er
> > output, etc., specifically with regard to u-boot.=C2=A0 I am not = sure if this
> > is still an issue or a concern, however.
>
> QUOTE
> author Emmanuel Vadot <manu@FreeBSD.org> 2021-05-11 18:27:14 +00= 00
> committer Emmanuel Vadot <manu@FreeBSD.org> 2021-05-11 20:22:54 = +0000
> commit 0d6e5081eb0080c4703f1c5cc69c34f38d9149b7 (patch)
> tree a22f954f3003c1361f4ea5a411e92759a80c9089 /sysutils/u-boot-master<= br> > parent c5fd1c2e186abb2e3209fa48d75d8dcdcda63f06 (diff)
> download ports-0d6e5081eb0080c4703f1c5cc69c34f38d9149b7.tar.gz
> ports-0d6e5081eb0080c4703f1c5cc69c34f38d9149b7.zip
>
> sysutils/u-boot-*: Remove ubldr support
>
> We have been using loader.efi on armv7 for a long time now. Remove sup= port for booting with ubldr and the needed patches that were never upstream= ed. While here add CONFIG_EFI_GRUB_ARM32_WORKAROUND=3Dy in the Fragment as = it's needed to have the cache flushed for us when loader.efi is started= .
> END QUOTE
>
>
> So: before 2021-Jun.
>

Noted.=C2=A0 Thank you for looking.

> > That said, I can take a look and see if we can package base.txz f= or
> > armv7, however I would like to do some archaeology work here to b= e sure
> > that the resultant output is not going to have unexpected behavio= r
> > because of the userland not matching 100% the target SoC.
>
>

Glen

--0000000000006e129005f75b8efb--