Re: Poudriere friendly armv7 relases

On Mon, Mar 20, 2023 at 2:28 PM Glen Barber <gjb@freebsd.org> wrote:

> On Mon, Mar 20, 2023 at 02:06:50PM -0600, Warner Losh wrote:
> > Greetings,
> >
> > Since it looks like we're going to retain at least armv7 for FreeBSD 14
> > (armv6 has been nominated for deprecation, but if it isn't deprecated,
> all
> > this applies to it).
> >
> > I'd like to start making at least the base.tgz, etc available for armv7.
> > This would allow us to create armv7 poduriere jails without building from
> > source.
> >
> > Is there some reason we're not doing this today? I know ISOs don't make a
> > lot of sense in the arm ecosystem, but having these artifacts would
> enable
> > poudriere binary install support.
> >
> > Comments?
> >
>
> Several.  :)
>
> I have looked into this in the past, and mhorne@ had even added some
> environment knobs to the way armv7 is built, however I later realized
> that it was not 1:1 compatible with how base.txz, etc., are generated
> for other architectures.
>
> 1) For other architectures, base.txz is result of the 'ftp' target in
>    /usr/src/release.
>
> 2) armv7 does not have an 'ftp' target.  (Well, it does not *disallow*
>    it, and probably should at the immediate moment, but it does blow
>    up.)
>
> 3) Most importantly, and the reason I stopped looking further into this,
>    we cannot run native armv7 binaries on an amd64 system (at least,
>    last I was aware).  Particularly, we can only actually use what is
>    listed in kern.supported_archs, at least without falling back to some
>    sort of emulation or wrapper support (such as qemu or the like).
>

actually we can. We can run them with bsd-user on amd64, and we
can run them on the arm64 hardware that has 32-bit support. This came
up in connection with my GSOC project to upstream everything, using
armv7 as the touchstone for success: the students that have looked into
it have had to build from sources.


> Back when armv6 and armv7 support was added using shell scripts instead
> of hooking into release/Makefile, having a base.txz did not make much
> sense because there were different environment variables that were
> passed into the resulting output, some of which affected the loader
> output, etc., specifically with regard to u-boot.  I am not sure if this
> is still an issue or a concern, however.
>

Yea, we've moved on from the ubldr needing a specific load address.
That was more for armv4 and armv5 boards we removed the support
for. Some of this bled over into armv7, but can safely be ignored now.


> That said, I can take a look and see if we can package base.txz for
> armv7, however I would like to do some archaeology work here to be sure
> that the resultant output is not going to have unexpected behavior
> because of the userland not matching 100% the target SoC.
>

These days, everything FreeBSD builds is identical on supported harndware.
We assume the boards can do UEFI after whatever weird thing they
need to get that going, and all that crazy has been relegated to the
uboot ports, and no longer intrudes into how we build stand (though some
crazy people might build stand for such platforms, it's so niche (eg a
couple
of routers that don't run our binaries ever) at this point that we
shouldn't worry
about it for releases).

We've evolved to a point where the benefit for being able to build poudriere
jails from release artifacts out-weights the couple of users being slightly
inconvenienced on their legacy platforms (and it hasn't been clear those
platforms will ever be updated to 14).

Warner