From nobody Mon Mar 20 20:38:34 2023 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PgRQt6M5kz40DJq for ; Mon, 20 Mar 2023 20:38:46 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-ed1-x52f.google.com (mail-ed1-x52f.google.com [IPv6:2a00:1450:4864:20::52f]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4PgRQt5Vr9z3jX5 for ; Mon, 20 Mar 2023 20:38:46 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-ed1-x52f.google.com with SMTP id x3so51774936edb.10 for ; Mon, 20 Mar 2023 13:38:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20210112.gappssmtp.com; s=20210112; t=1679344725; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=tsC+OQOp5V2yJ1f/bWqX08JrD3mOJHVLLq7JiwfcqGk=; b=aCOTYemyWTDym3fbk0qQWkdxMBEgxY+uOhDG0gi8PiBCmfzIkC4tnEqzUQqvj9fitI 2JxBYJNg301xROiPGkRwBhRhpsbFt3R/YhBhsCYC02sWGRwn0vg6flvLmNIwZGSipihL o9yyqbq6iixTZnrArd6f8jhO6IqaNgwaU1/8gMVQtxJ0kFO+MRa7XvDMJfWwfuW8ygMG 18mj2yH8HG4lGAlRC6yGBMdcyctQvc7O4d/jfpaPlqe23gMf7MR+wzCWOXJFgR3ERsT4 ab2xBACW1sfE/UyTt+Hu9LNKQaQfQWoGxzDeoxEth4uhFij/xve0Xu5+0R1M7J65k9NR UI6Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679344725; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=tsC+OQOp5V2yJ1f/bWqX08JrD3mOJHVLLq7JiwfcqGk=; b=FXc+43ZZu64hKYBn8GtQQenxJZgzwqRizw/P7gL6HZ2FU3Dvo1/9uBXtLIyKaV5VM2 ABv2tK8CS6D89m0f5dgVRCoIpBvclCxMK86BBCtTxW4bfI7a8EIfMIABYUIAoq7hHwU8 /lUul4j3EsLzjrAaJOPNrP6xCGiE3e1VobKbAvznqHt8Djp4cTEX0m28PVzau+rhMNNK /YOOaIdKZHhyAxe1JX7eDVXp/8/Gh/m3Fv5Hqs/DTa6hI7YUvfuvyZW0UL27rtoyj7JY ZoJWYXHJLOCdV+bltzbA6XwJ7AR1sXF2esaRD3J9jEL44ATlk225/f7v0SS0Pv4ej55I R79w== X-Gm-Message-State: AO0yUKUHOh0DDaSTBO4n+nF7FZDtn7pa3D+FiuGn30iTe4+imCrZaH5d nlw74mhWTeB0kq95eWxflN/L2XEvTuzlAGzy96GCYBt7R22c4T59zDniWg== X-Google-Smtp-Source: AK7set+D432yaVSYQrL9Ne146fQGDmhY5wZb3zh+geRgNvMLPIgRanLBNyfcihohIf/C/A7C4vOuSb8YDn0WXvGDZio= X-Received: by 2002:a50:ce45:0:b0:4fb:80cf:898b with SMTP id k5-20020a50ce45000000b004fb80cf898bmr435842edj.7.1679344725069; Mon, 20 Mar 2023 13:38:45 -0700 (PDT) List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org MIME-Version: 1.0 References: <20230320202847.GW2347@FreeBSD.org> In-Reply-To: <20230320202847.GW2347@FreeBSD.org> From: Warner Losh Date: Mon, 20 Mar 2023 14:38:34 -0600 Message-ID: Subject: Re: Poudriere friendly armv7 relases To: Glen Barber Cc: "freebsd-arm@freebsd.org" Content-Type: multipart/alternative; boundary="000000000000cb296f05f75ae901" X-Rspamd-Queue-Id: 4PgRQt5Vr9z3jX5 X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N --000000000000cb296f05f75ae901 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Mon, Mar 20, 2023 at 2:28=E2=80=AFPM Glen Barber wrote= : > On Mon, Mar 20, 2023 at 02:06:50PM -0600, Warner Losh wrote: > > Greetings, > > > > Since it looks like we're going to retain at least armv7 for FreeBSD 14 > > (armv6 has been nominated for deprecation, but if it isn't deprecated, > all > > this applies to it). > > > > I'd like to start making at least the base.tgz, etc available for armv7= . > > This would allow us to create armv7 poduriere jails without building fr= om > > source. > > > > Is there some reason we're not doing this today? I know ISOs don't make= a > > lot of sense in the arm ecosystem, but having these artifacts would > enable > > poudriere binary install support. > > > > Comments? > > > > Several. :) > > I have looked into this in the past, and mhorne@ had even added some > environment knobs to the way armv7 is built, however I later realized > that it was not 1:1 compatible with how base.txz, etc., are generated > for other architectures. > > 1) For other architectures, base.txz is result of the 'ftp' target in > /usr/src/release. > > 2) armv7 does not have an 'ftp' target. (Well, it does not *disallow* > it, and probably should at the immediate moment, but it does blow > up.) > > 3) Most importantly, and the reason I stopped looking further into this, > we cannot run native armv7 binaries on an amd64 system (at least, > last I was aware). Particularly, we can only actually use what is > listed in kern.supported_archs, at least without falling back to some > sort of emulation or wrapper support (such as qemu or the like). > actually we can. We can run them with bsd-user on amd64, and we can run them on the arm64 hardware that has 32-bit support. This came up in connection with my GSOC project to upstream everything, using armv7 as the touchstone for success: the students that have looked into it have had to build from sources. > Back when armv6 and armv7 support was added using shell scripts instead > of hooking into release/Makefile, having a base.txz did not make much > sense because there were different environment variables that were > passed into the resulting output, some of which affected the loader > output, etc., specifically with regard to u-boot. I am not sure if this > is still an issue or a concern, however. > Yea, we've moved on from the ubldr needing a specific load address. That was more for armv4 and armv5 boards we removed the support for. Some of this bled over into armv7, but can safely be ignored now. > That said, I can take a look and see if we can package base.txz for > armv7, however I would like to do some archaeology work here to be sure > that the resultant output is not going to have unexpected behavior > because of the userland not matching 100% the target SoC. > These days, everything FreeBSD builds is identical on supported harndware. We assume the boards can do UEFI after whatever weird thing they need to get that going, and all that crazy has been relegated to the uboot ports, and no longer intrudes into how we build stand (though some crazy people might build stand for such platforms, it's so niche (eg a couple of routers that don't run our binaries ever) at this point that we shouldn't worry about it for releases). We've evolved to a point where the benefit for being able to build poudrier= e jails from release artifacts out-weights the couple of users being slightly inconvenienced on their legacy platforms (and it hasn't been clear those platforms will ever be updated to 14). Warner --000000000000cb296f05f75ae901 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


=
On Mon, Mar 20, 2023 at 2:28=E2=80=AF= PM Glen Barber <gjb@freebsd.org&g= t; wrote:
On Mon= , Mar 20, 2023 at 02:06:50PM -0600, Warner Losh wrote:
> Greetings,
>
> Since it looks like we're going to retain at least armv7 for FreeB= SD 14
> (armv6 has been nominated for deprecation, but if it isn't depreca= ted, all
> this applies to it).
>
> I'd like to start making at least the base.tgz, etc available for = armv7.
> This would allow us to create armv7 poduriere jails without building f= rom
> source.
>
> Is there some reason we're not doing this today? I know ISOs don&#= 39;t make a
> lot of sense in the arm ecosystem, but having these artifacts would en= able
> poudriere binary install support.
>
> Comments?
>

Several.=C2=A0 :)

I have looked into this in the past, and mhorne@ had even added some
environment knobs to the way armv7 is built, however I later realized
that it was not 1:1 compatible with how base.txz, etc., are generated
for other architectures.

1) For other architectures, base.txz is result of the 'ftp' target = in
=C2=A0 =C2=A0/usr/src/release.

2) armv7 does not have an 'ftp' target.=C2=A0 (Well, it does not *d= isallow*
=C2=A0 =C2=A0it, and probably should at the immediate moment, but it does b= low
=C2=A0 =C2=A0up.)

3) Most importantly, and the reason I stopped looking further into this, =C2=A0 =C2=A0we cannot run native armv7 binaries on an amd64 system (at lea= st,
=C2=A0 =C2=A0last I was aware).=C2=A0 Particularly, we can only actually us= e what is
=C2=A0 =C2=A0listed in kern.supported_archs, at least without falling back = to some
=C2=A0 =C2=A0sort of emulation or wrapper support (such as qemu or the like= ).

actually we can. We can run them wit= h=C2=A0bsd-user on amd64, and we
can run them on the arm64 hardwa= re that has 32-bit support. This came
up in connection with my GS= OC project to upstream everything, using
armv7 as the touchstone = for success: the students that have looked into
it have had to bu= ild from sources.
=C2=A0
Back when armv6 and armv7 support was added using shell scripts instead
of hooking into release/Makefile, having a base.txz did not make much
sense because there were different environment variables that were
passed into the resulting output, some of which affected the loader
output, etc., specifically with regard to u-boot.=C2=A0 I am not sure if th= is
is still an issue or a concern, however.

Yea, we've moved on from the ubldr needing a specific load address.
That was more for armv4 and armv5 boards we removed the support
for. Some of this bled over into armv7, but can safely be ignored n= ow.
=C2=A0
That said, I can take a look and see if we can package base.txz for
armv7, however I would like to do some archaeology work here to be sure
that the resultant output is not going to have unexpected behavior
becau= se of the userland not matching 100% the target SoC.
<= br>
These days, everything FreeBSD builds is identical on support= ed harndware.
We assume the boards can do UEFI after whatever wei= rd thing they
need to get that going, and all that crazy has been= relegated to the
uboot ports, and no longer intrudes into how we= build stand (though some
crazy people might build stand for such= platforms, it's so niche (eg a couple
of routers that don= 9;t run our binaries ever) at this point that we shouldn't worry
<= div>about it for releases).

We've evolved to a= point where the benefit for being able to build poudriere
jails = from release artifacts out-weights the couple of users being slightly
=
inconvenienced=C2=A0on their legacy platforms (and it hasn't been = clear those
platforms will ever be updated to 14).

=
Warner
--000000000000cb296f05f75ae901--