From nobody Fri Jan 12 17:57:30 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TBTlV1rVCz57G57 for ; Fri, 12 Jan 2024 17:57:42 +0000 (UTC) (envelope-from dfr@rabson.org) Received: from mail-yb1-xb30.google.com (mail-yb1-xb30.google.com [IPv6:2607:f8b0:4864:20::b30]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4TBTlT1NcJz4m9s for ; Fri, 12 Jan 2024 17:57:41 +0000 (UTC) (envelope-from dfr@rabson.org) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=rabson-org.20230601.gappssmtp.com header.s=20230601 header.b="QAjxWG/M"; dmarc=none; spf=pass (mx1.freebsd.org: domain of dfr@rabson.org designates 2607:f8b0:4864:20::b30 as permitted sender) smtp.mailfrom=dfr@rabson.org Received: by mail-yb1-xb30.google.com with SMTP id 3f1490d57ef6-dbed85ec5b5so5495154276.3 for ; Fri, 12 Jan 2024 09:57:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rabson-org.20230601.gappssmtp.com; s=20230601; t=1705082260; x=1705687060; darn=freebsd.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=qM4tWqQDfZoLSufw8tZE+YKA3Igv9zj1c8H834BTqlQ=; b=QAjxWG/Me1hJv05m/XakLXy7ctrZckQZepYo7duS2LB6RxJjRhU/WB54RIbpObbBmT C2ykBV7fxFkoY0DV0C91QRtSPlY9C9cBW1oqp38+XrL/7uipb62mRlXk+9hNtPHVe/0s jMusY3myShbjcyw5EYrm8Xnic72MQodObtBAc5HAj5sxeclNHnaKK+4BAP4ZaSizJBHc vNoe7WZDRdqZVlj15E3/0FuAx9yeutwt8MrDDzTyGfxVJBLdgrA663eG9iVvNZGfFpjs 42KPh69JSDTPCQ0uVGlj0Tht4N7OIaIRsNGthDh5tIwGk7Y8nryMVQy2fR8rl+7gm3gc UQHA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1705082260; x=1705687060; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=qM4tWqQDfZoLSufw8tZE+YKA3Igv9zj1c8H834BTqlQ=; b=S9zdveJz4vULFg3oR2poWPGkSMas/XUnq53WKbS/pM7K84HZzC9SGnS/c1ZuRT4pY9 uBJJry+1Yz3rEm1Q9sbsIwWqD5PCREjcDwdcRFcq8OaQ3FtrJAiWWJy3FkWMUgC9z6ow ApUsJaAw12e+arZ0jKaAndZmCjso+lx9uMbFKtQM2O6MZUb5jcEgkwaWHfw+occ0LJwp m2gQF/bjJwfqp9XViybJr0kGRtShfCJXCZM5+IgiH9ic9rWq88tUkBaY/yAkTvprPoHO oXodZaF5RRgQfOSBA35VPHglDLRnEBpl8ztj5vsuzJw8tsAElF2Zy7+wzsdGj/7b2Q7T W4Jw== X-Gm-Message-State: AOJu0Yz1P8Ho+iH92HU2aPPbToF2i15o9Qmbobg0nze7i4skE87bRKtT J6cgSa11v1rz58bMOAcVHGALjwcCLZHh2gtJOJgtqgz9iXIf1zzqRQglmG9unY3gOA== X-Google-Smtp-Source: AGHT+IHvML+kf0qsvmuK4n9CWqZPIGtlANuO82waX+AGnx1t5x6eJSzSOj68wtbMUFj4yJCo1d0uHbK+qmK3xfb1FUI= X-Received: by 2002:a25:ae48:0:b0:dbd:ac49:223 with SMTP id g8-20020a25ae48000000b00dbdac490223mr871043ybe.90.1705082259965; Fri, 12 Jan 2024 09:57:39 -0800 (PST) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 References: <3CB904C2-D983-4EF7-84D3-6BDED0700B08.ref@yahoo.com> <3CB904C2-D983-4EF7-84D3-6BDED0700B08@yahoo.com> In-Reply-To: <3CB904C2-D983-4EF7-84D3-6BDED0700B08@yahoo.com> From: Doug Rabson Date: Fri, 12 Jan 2024 17:57:30 +0000 Message-ID: Subject: Re: 15 & 14: ram_attach vs. its using regions_to_avail vs. "bus_alloc_resource" can lead to: panic("ram_attach: resource %d failed to attach", rid) To: Mark Millard Cc: Current FreeBSD , FreeBSD-STABLE Mailing List Content-Type: multipart/alternative; boundary="0000000000006b1e9f060ec36675" X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.50 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.999]; R_DKIM_ALLOW(-0.20)[rabson-org.20230601.gappssmtp.com:s=20230601]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; FREEFALL_USER(0.00)[dfr]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::b30:from]; MIME_TRACE(0.00)[0:+,1:+,2:~]; MISSING_XM_UA(0.00)[]; ARC_NA(0.00)[]; RCVD_COUNT_ONE(0.00)[1]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; TO_MATCH_ENVRCPT_SOME(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; DMARC_NA(0.00)[rabson.org]; TO_DN_ALL(0.00)[]; RCVD_TLS_LAST(0.00)[]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; FREEMAIL_TO(0.00)[yahoo.com]; DKIM_TRACE(0.00)[rabson-org.20230601.gappssmtp.com:+] X-Rspamd-Queue-Id: 4TBTlT1NcJz4m9s --0000000000006b1e9f060ec36675 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Sat, 30 Sept 2023 at 08:47, Mark Millard wrote: > ram_attach is based on regions_to_avail but that is a problem for > its later bus_alloc_resource use --and that can lead to: > > panic("ram_attach: resource %d failed to attach", rid); > > Unfortunately, the known example is use of EDK2 on RPi4B > class systems, not what is considered the supported way. > The panic happens for main [so: 15] and will happen once > the cortex-a72 handling in 14.0-* is in a build fixed by: > > =E2=80=A2 git: 906bcc44641d - releng/14.0 - arm64: Fix errata workaro= unds that > depend on smccc Andrew Turner > > The lack of the fix leads to an earlier panic as stands. > > > sys/kern/subr_physmem.c 's regions_to_avail is based on ignoring > phys_avail and using only hwregions and exregions. In other words, > in part: > > * Initially dump_avail and phys_avail are identical. Boot time memory > * allocations remove extents from phys_avail that may still be included > * in dumps. > > This means that early, dedicated memory allocations are treated > as available for general use by regions_to_avail . The distinction > is visible in the boot -v output in that: > > real memory =3D 3138154496 (2992 MB) > Physical memory chunk(s): > 0x00000000200000 - 0x0000002b7fffff, 727711744 bytes (177664 pages) > 0x0000002ce3a000 - 0x0000003385ffff, 111304704 bytes (27174 pages) > 0x000000338c0000 - 0x000000338c6fff, 28672 bytes (7 pages) > 0x00000033a30000 - 0x00000036efffff, 55377920 bytes (13520 pages) > 0x000000372e0000 - 0x0000003b2fffff, 67239936 bytes (16416 pages) > 0x00000040000000 - 0x000000bb3dcfff, 2067648512 bytes (504797 pages) > avail memory =3D 3027378176 (2887 MB) > > does not list the wider: > > 0x00000040000000 - 0x000000bfffffff > > because of phys_avail . But the earlier dump based on hwregions and > exregions shows: > > Physical memory chunk(s): > 0x001d0000 - 0x001effff, 0 MB ( 32 pages) > 0x00200000 - 0x338c6fff, 822 MB ( 210631 pages) > 0x33920000 - 0x3b2fffff, 121 MB ( 31200 pages) > 0x40000000 - 0xbfffffff, 2048 MB ( 524288 pages) > Excluded memory regions: > 0x001d0000 - 0x001effff, 0 MB ( 32 pages) NoAlloc > 0x2b800000 - 0x2ce39fff, 22 MB ( 5690 pages) NoAlloc > 0x33860000 - 0x338bffff, 0 MB ( 96 pages) NoAlloc > 0x33920000 - 0x33a2ffff, 1 MB ( 272 pages) NoAlloc > 0x36f00000 - 0x372dffff, 3 MB ( 992 pages) NoAlloc > > which indicates: > > 0x40000000 - 0xbfffffff > > is available as far as it is concerned. > > (Note some code works/displays in terms of: 0x40000000 - 0xc0000000 > instead.) > > For aarch64 , sys/arm64/arm64/nexus.c has a nexus_alloc_resource > that is used as bus_alloc_resource . It ends up rejecting the > RPi4B boot via using the result of the call in ram_attach: > > if (bus_alloc_resource(dev, SYS_RES_MEMORY, &rid, start, > end, > end - start, 0) =3D=3D NULL) > panic("ram_attach: resource %d failed to attach", > rid); > > as shown by the just-prior start/end pair sequence messages: > > ram0: reserving memory region: 200000-2b800000 > ram0: reserving memory region: 2ce3a000-33860000 > ram0: reserving memory region: 338c0000-338c7000 > ram0: reserving memory region: 33a30000-36f00000 > ram0: reserving memory region: 372e0000-3b300000 > ram0: reserving memory region: 40000000-c0000000 > panic: ram_attach: resource 5 failed to attach > > I do not see anything about this that looks inherently RPi* > specific for possibly ending up with an analogous panic. So > I expect the example is sufficient context to identify a > problem is present, despite EDK2 use not being normal for > RPi4B's and the like as far as FreeBSD is concerned. > I'm not quite clear why phys_avail changes and why that is triggered by the 906bcc44641d commit. I'm wondering if it makes sense to arrange for ram_attach to happen before acpi, e.g. using BUS_PASS_ORDER_FIRST? Doug. --0000000000006b1e9f060ec36675 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


=
On Sat, 30 Sept 2023 at 08:47, Mark M= illard <marklmi@yahoo.com> w= rote:
ram_attach is based on regions_to_avail but= that is a problem for
its later bus_alloc_resource use --and that can lead to:

panic("ram_attach: resource %d failed to attach", rid);

Unfortunately, the known example is use of EDK2 on RPi4B
class systems, not what is considered the supported way.
The panic happens for main [so: 15] and will happen once
the cortex-a72 handling in 14.0-* is in a build fixed by:

=C2=A0 =C2=A0 =E2=80=A2 git: 906bcc44641d - releng/14.0 - arm64: Fix errata= workarounds that depend on smccc Andrew Turner

The lack of the fix leads to an earlier panic as stands.


sys/kern/subr_physmem.c 's regions_to_avail is based on ignoring
phys_avail and using only hwregions and exregions. In other words,
in part:

=C2=A0* Initially dump_avail and phys_avail are identical.=C2=A0 Boot time = memory
=C2=A0* allocations remove extents from phys_avail that may still be includ= ed
=C2=A0* in dumps.

This means that early, dedicated memory allocations are treated
as available for general use by regions_to_avail . The distinction
is visible in the=C2=A0 boot -v output in that:

real memory=C2=A0 =3D 3138154496 (2992 MB)
Physical memory chunk(s):
0x00000000200000 - 0x0000002b7fffff, 727711744 bytes (177664 pages)
0x0000002ce3a000 - 0x0000003385ffff, 111304704 bytes (27174 pages)
0x000000338c0000 - 0x000000338c6fff, 28672 bytes (7 pages)
0x00000033a30000 - 0x00000036efffff, 55377920 bytes (13520 pages)
0x000000372e0000 - 0x0000003b2fffff, 67239936 bytes (16416 pages)
0x00000040000000 - 0x000000bb3dcfff, 2067648512 bytes (504797 pages)
avail memory =3D 3027378176 (2887 MB)

does not list the wider:

0x00000040000000 - 0x000000bfffffff

because of phys_avail . But the earlier dump based on hwregions and
exregions shows:

Physical memory chunk(s):
=C2=A0 0x001d0000 - 0x001effff,=C2=A0 =C2=A0 =C2=A00 MB (=C2=A0 =C2=A0 =C2= =A032 pages)
=C2=A0 0x00200000 - 0x338c6fff,=C2=A0 =C2=A0822 MB ( 210631 pages)
=C2=A0 0x33920000 - 0x3b2fffff,=C2=A0 =C2=A0121 MB (=C2=A0 31200 pages)
=C2=A0 0x40000000 - 0xbfffffff,=C2=A0 2048 MB ( 524288 pages)
Excluded memory regions:
=C2=A0 0x001d0000 - 0x001effff,=C2=A0 =C2=A0 =C2=A00 MB (=C2=A0 =C2=A0 =C2= =A032 pages) NoAlloc
=C2=A0 0x2b800000 - 0x2ce39fff,=C2=A0 =C2=A0 22 MB (=C2=A0 =C2=A05690 pages= ) NoAlloc
=C2=A0 0x33860000 - 0x338bffff,=C2=A0 =C2=A0 =C2=A00 MB (=C2=A0 =C2=A0 =C2= =A096 pages) NoAlloc
=C2=A0 0x33920000 - 0x33a2ffff,=C2=A0 =C2=A0 =C2=A01 MB (=C2=A0 =C2=A0 272 = pages) NoAlloc
=C2=A0 0x36f00000 - 0x372dffff,=C2=A0 =C2=A0 =C2=A03 MB (=C2=A0 =C2=A0 992 = pages) NoAlloc

which indicates:

=C2=A0 0x40000000 - 0xbfffffff

is available as far as it is concerned.

(Note some code works/displays in terms of: 0x40000000 - 0xc0000000
instead.)

For aarch64 , sys/arm64/arm64/nexus.c has a nexus_alloc_resource
that is used as bus_alloc_resource . It ends up rejecting the
RPi4B boot via using the result of the call in ram_attach:

=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 if (bus_alloc_resou= rce(dev, SYS_RES_MEMORY, &rid, start, end,
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 end -= start, 0) =3D=3D NULL)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 panic("ram_attach: resource %d failed to attach", rid)= ;

as shown by the just-prior start/end pair sequence messages:

ram0: reserving memory region:=C2=A0 =C2=A0200000-2b800000
ram0: reserving memory region:=C2=A0 =C2=A02ce3a000-33860000
ram0: reserving memory region:=C2=A0 =C2=A0338c0000-338c7000
ram0: reserving memory region:=C2=A0 =C2=A033a30000-36f00000
ram0: reserving memory region:=C2=A0 =C2=A0372e0000-3b300000
ram0: reserving memory region:=C2=A0 =C2=A040000000-c0000000
panic: ram_attach: resource 5 failed to attach

I do not see anything about this that looks inherently RPi*
specific for possibly ending up with an analogous panic. So
I expect the example is sufficient context to identify a
problem is present, despite EDK2 use not being normal for
RPi4B's and the like as far as FreeBSD is concerned.

I'm not quite clear why phys_avail changes and why th= at is triggered by the 906bcc44641d commit. I'm wondering if it makes s= ense to arrange for ram_attach to happen before acpi, e.g. using BUS_PASS_O= RDER_FIRST?

Doug.
=C2=A0
--0000000000006b1e9f060ec36675--