From nobody Tue Feb 13 01:57:54 2024 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TYkxd5YHtz59Tyv for ; Tue, 13 Feb 2024 01:58:13 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic305-21.consmr.mail.gq1.yahoo.com (sonic305-21.consmr.mail.gq1.yahoo.com [98.137.64.84]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4TYkxc37Kpz4sqx for ; Tue, 13 Feb 2024 01:58:12 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=UQpVBNSh; dmarc=pass (policy=reject) header.from=yahoo.com; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.64.84 as permitted sender) smtp.mailfrom=marklmi@yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1707789490; bh=yFdUWVGnKaC0hJq5OciUhv7RAE1dQWRhgaCkqTdoQmg=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=UQpVBNShUOYbXGYAon+rZcyxmqG1Vy0nYUQrWeXJ9MrljXEmBJG5QHc51ZKBJmdX/Qfw8cTL0hDU0ra35dIPJ/PVGJKtoOJMskaIDF8F5UzttNCkW+Mik2r3bk7FH/UqdD55e6H4m8pv6aTahZFnIE/jHAhtNUC/gzHpX1J4U4y2bYSc4PSfgpV7/81PhPviXFbAczHJFqUeX6DUBuwzExRUJiZVluFvtcMS72aoKI+39g4UGKFpsqm1WihBXxiSj7Bsx7iw94fj9RLpsn0P4bp3KfnSwOTlSAUk+mIwhr7rgec8S1XiKUcXB3mTbaYPzz6BM2/pYsX8Cj64dmyF2g== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1707789490; bh=UDt+I4D1OB/a7Jn4SUOBZVGxyvjYDHrK4/acf3Gqpqd=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=FOacDyF+Th9Fv+zMUOYQIGUMNuOD+Vjzn13XiigrLTUQw4HixqeL5q4vPUxMHsB+hlHCYS9laJ4w+2H8BqPFBW8Hu3dn6HJJx2S1iYa2Av0dXi38FlR4HwOBrf14eL+c9MfeYBRnZR1aqfPHtnt/sHiux30i7fqY++CAHs23MgnQA+UxL6RqY0sFc9hWfKJiGTX2WOiQ4sJETj5GASDOn983J2fHZL6lcdpEWpsHffxP1HKqnZku0a3+njig0i54StXjPngtVnDENtvVX8nlhBB1+8sqH5EVnexPo+XiPXhRWH9Fb7tbPbIk+z6O5zQHXIwzAolHFMwRFjbi7qFKwQ== X-YMail-OSG: WCeifLsVM1lRuZdxXxFjvlwTzuKew4d5K85ZvXCj5sr5uhrhzXs8C1dtuI4ulJa LH8GLPqBXT0u._3gi09JEL_445n_7WdGxbXdRB26iaR5AfQSv5cZQ0NBwpRUyREvEFOh3Bc_GIxi IchSR97jQf9z3B1wouusBqwOQLQUh8U3Lq.yluPn0PuMWGZ.be1o2CY28jg2Oae2Og4E1E5YP5Ys ifYxua6zleuOjaV3KGzvsxgA0lUvrON9ZQwfzloqkRHlFGyFkmZ3YMZcr82D5NJhh93kWvmAa_fY uF25qpuunxL8eRAoPrVj.1oUBH2XHHcXe7rB0HM8FElx_25x.zUuqSwrUlVEKTku60RED5QI5ibo s3Zgs66Yc2tShn5_Gb3HtZtHQZZMLorkNYCnaXnPnLRfgLt2KgDrkJ.c3KGo8Hx6fksgPLUfAP_5 oE.Hwf4nIGEpBwqFxPFvVsOocY5MfmaQanbM7vfpVgRaFlvxLk48XXcqil9JUmlVlUV7CIGZaf.1 PyVMznuCDUy1Aaq0rAZCackajthIKvlKrMAJNRSRpeMGQn2Dy7rZ.RU1YSGBe1P7Dmaf8qHzFM4E qWL6hr3ERwtgmsDoL5uLEbbAnmh9C97rqV2Xi56Wh_Gu9Ctj07YTOJsNq2ts8rxDfgqdHqhDOmh3 9dDeFv9AJf0zGFXCbww06WGkuFODmyPqs3JCnkl8OjSubKPVEnjtYLgG0CMJCG6ufdnCkcv1yc2K zPmZvLTzVLeUDhR2tmJ4y2TcesCLG.nu3ICGtzbJ.WPhE1hJRE6GBr3cNqLtfh30ySz._4zawfJX i3IcbVeXTgWAyh9sXf8vj11DrRuC1jT2oLxkSI3PjExrWg5D.mdMNqW0bo1QF4jg1oLFp8mF9hD1 B3CxVXPmAAZ2CnnK0BoRYWh2Qpb.C8WnOEVJs0qvnEpKe3Nl.goBLEKImqN2K3.sLxOtR6sVQcoN wyh9N94aMr1WrUe8hPfiNooJBhhwJyh86nXXVXSwCa50QPZabymW0UAKy97gEosWjBgUelAVYcZx NMUs1TsGYHlB7oQwRR.P6BK6p2ZJ0sCuHuHJFrOdFvCWHUo0ywSIM4JqQLpAIYSdGUZ3vGh5gU.B COUdfU2bz2v44h803u3Q.AoK77y1RlJD5zSIupKC6VpB1bXYdAM.DuaPz4zwLT3PhaKXMupvOiEy GoSgfENF1dC.iYcT3uyQJy1hDzPgSxuiamPhvhg_d6NAqJE2EVHR2MdFHZA6mfgf5HJR1eanV3nm rhFoXIxtkwbYb0O4thO.F5dhyc6zwCIjwYv8xB40nw.qb_fREBFGsLZanvX6UTXD9DIOruRWa1T. AlXzUZuq4CFvfQXZR7oGgeuR4H58HqQJB1DanbB6wWB.NxQ5WWFYfzPHSaEwQGmNZx_H786LzXJK cqOEsVmnGnwtEOxRH6xOVjagJiW5WWzglZinIvk6MFVBOeDM9XUJQnZ4ssiujoEvzlT1Xxtm.TQG xXKNj8XxxfWaomRJg_MUdfES0.4iiAWqgRBXCKCbX_LsqLQDxlN_jh1yHLS8JttBGwSqijlprYXo etvS5KJaagHP9JzIuGAPHdqiKCTqsr0aA.eCPb7NIJH8YzybidPl.v1m.6imUUGO5iNEYPS_wSTf 6UFKx79gfD4e9XNi8JWUbP.J8W.Xx.9uUjlU5gJVGfjw_vPBuz2RlA.MqakbTMIJlMSD1NXoKn2S iOWAfrXR1dtZmJVmYC61FXZ2vIPPABSAHDeRFyI_BRd8dhBt5V59lDvrhYR3hJWXiBLyXNR5Ybpn OJ_fP07Fs7HD3Huwpst5er1RJ33a1xxOdvKG6HRr4Oo_g19pPJ_Fsj8iIS4.0CCNBGKUvPvtY6a. 8VgYQ0CqelObqb8JP6j_c5UEiDYePlH2VC7R.Nyjz0hRibtGXKqGdJ6VgAOxOe_9J0wj60puJUY. vRhZX1pHTPdXGk0OB25SxShV.Qa3S4Tp5TApJ7HPNB2gNlBJBE7jkrJn.AcZxoUCKHII4hei5843 ifzYC_4tatcK9bpuNlcRPl_sOLtSI0EZPeSn5R1rvY_tembs7Zj_3rnMoix0qIhzpwgZOu1N85Nw rYciNCmQtMhk2diGU5POsuiBCmSWtQ593OlX40TNf22r1GIZrQfgdoJnJU.pvlPdWpGgz2jslI81 6qrAGmXJJkc_7Y8Z1rO2caQ4nQD5N9oh_Q32sEUukh6tOcBLvquUu1XzYI_HBX29FtlKKwx9abyu yJV.5TITw2s6MbyOpD0xRhIwTmgC7udr7L1AsXLD1XwDrbTK_Mm.fxNqTkT__zz_bvmlNSPkG3x8 88GE- X-Sonic-MF: X-Sonic-ID: db472841-ae12-47c3-a723-2fc5cedf1274 Received: from sonic.gate.mail.ne1.yahoo.com by sonic305.consmr.mail.gq1.yahoo.com with HTTP; Tue, 13 Feb 2024 01:58:10 +0000 Received: by hermes--production-gq1-5c57879fdf-27p5r (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID ae8256dfa2d8bb9483f05df3dc891bec; Tue, 13 Feb 2024 01:58:05 +0000 (UTC) Content-Type: text/plain; charset=us-ascii List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.400.31\)) Subject: Re: Recent commits reject RPi4B booting: pcib0 vs. pcib1 "rman_manage_region: request" leads to panic From: Mark Millard In-Reply-To: <3A145420-399D-4EBD-9FF4-18924908AB1D@yahoo.com> Date: Mon, 12 Feb 2024 17:57:54 -0800 Cc: FreeBSD ARM List , Current FreeBSD , Warner Losh Content-Transfer-Encoding: quoted-printable Message-Id: <1298DF9C-0F82-4567-8E81-7332A608C7FC@yahoo.com> References: <76AB969F-5BC5-4116-8AF4-3ED2CABEBBA5.ref@yahoo.com> <76AB969F-5BC5-4116-8AF4-3ED2CABEBBA5@yahoo.com> <1F704317-FDB8-4BDA-8A67-61CF48794DFE@yahoo.com> <9AFDF067-96E4-4E67-90D2-F40DAF3F138F@yahoo.com> <4C279710-5F88-4295-B1A4-7C395F3587E5@yahoo.com> <3A145420-399D-4EBD-9FF4-18924908AB1D@yahoo.com> To: John Baldwin X-Mailer: Apple Mail (2.3774.400.31) X-Spamd-Bar: --- X-Spamd-Result: default: False [-4.00 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MIME_GOOD(-0.10)[text/plain]; DKIM_TRACE(0.00)[yahoo.com:+]; TO_DN_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; RCVD_TLS_LAST(0.00)[]; FREEMAIL_FROM(0.00)[yahoo.com]; MIME_TRACE(0.00)[0:+]; RCPT_COUNT_THREE(0.00)[4]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; MLMMJ_DEST(0.00)[freebsd-arm@freebsd.org]; APPLE_MAILER_COMMON(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.64.84:from]; RCVD_IN_DNSWL_NONE(0.00)[98.137.64.84:from] X-Rspamd-Queue-Id: 4TYkxc37Kpz4sqx On Feb 12, 2024, at 16:36, Mark Millard wrote: > On Feb 12, 2024, at 16:10, Mark Millard wrote: >=20 >> On Feb 12, 2024, at 12:00, Mark Millard wrote: >>=20 >>> [Gack: I was looking at the wrong vintage of source code, predating >>> your changes: wrong system used.] >>>=20 >>>=20 >>> On Feb 12, 2024, at 10:41, Mark Millard wrote: >>>=20 >>>> On Feb 12, 2024, at 09:32, John Baldwin wrote: >>>>=20 >>>>> On 2/9/24 8:13 PM, Mark Millard wrote: >>>>>> Summary: >>>>>> pcib0: mem = 0x7d500000-0x7d50930f irq 80,81 on simplebus2 >>>>>> pcib0: parsing FDT for ECAM0: >>>>>> pcib0: PCI addr: 0xc0000000, CPU addr: 0x600000000, Size: = 0x40000000 >>>>>> . . . >>>>>> rman_manage_region: request: start = 0x600000000, end 0x6000fffff >>>>>> panic: Failed to add resource to rman >>>>>=20 >>>>> Hmmm, I suspect this is due to the way that bus_translate_resource = works which is >>>>> fundamentally broken. It rewrites the start address of a resource = in-situ instead >>>>> of keeping downstream resources separate from the upstream = resources. For example, >>>>> I don't see how you could ever release a resource in this design = without completely >>>>> screwing up your rman. That is, I expect trying to detach a PCI = device behind a >>>>> translating bridge that uses the current approach should corrupt = the allocated >>>>> resource ranges in an rman long before my changes. >>>>>=20 >>>>> That said, that doesn't really explain the panic. Hmm, the panic = might be because >>>>> for PCI bridge windows the driver now passes RF_ACTIVE and the = bus_translate_resource >>>>> hack only kicks in the activate_resource method of = pci_host_generic.c. >>>>>=20 >>>>>> Detail: >>>>>> . . . >>>>>> pcib0: mem = 0x7d500000-0x7d50930f irq 80,81 on simplebus2 >>>>>> pcib0: parsing FDT for ECAM0: >>>>>> pcib0: PCI addr: 0xc0000000, CPU addr: 0x600000000, Size: = 0x40000000 >>>>>=20 >>>>> This indicates this is a translating bus. >>>>>=20 >>>>>> pcib1: irq 91 at device 0.0 on pci0 >>>>>> rman_manage_region: request: start 0x1, end = 0x1 >>>>>> pcib0: rman_reserve_resource: start=3D0xc0000000, end=3D0xc00fffff,= count=3D0x100000 >>>>>> rman_reserve_resource_bound: request: [0xc0000000, = 0xc00fffff], length 0x100000, flags 102, device pcib1 >>>>>> rman_reserve_resource_bound: trying 0xffffffff = <0xc0000000,0xfffff> >>>>>> considering [0xc0000000, 0xffffffff] >>>>>> truncated region: [0xc0000000, 0xc00fffff]; size 0x100000 = (requested 0x100000) >>>>>> candidate region: [0xc0000000, 0xc00fffff], size 0x100000 >>>>>> allocating from the beginning >>>>>> rman_manage_region: request: start = 0x600000000, end 0x6000fffff >>=20 >> What you later typed does not match: >>=20 >> 0x600000000 >> 0x6000fffff >>=20 >> You later typed: >>=20 >> 0x60000000 >> 0x600fffffff >>=20 >> This seems to have lead to some confusion from using the >> wrong figure(s). >>=20 >>>>> The fact that we are trying to reserve the CPU addresses in the = rman is because >>>>> bus_translate_resource rewrote the start address in the resource = after it was allocated. >>>>>=20 >>>>> That said, I can't see why rman_manage_region would actually fail. = At this point the >>>>> rman is empty (this is the first call to rman_manage_region for = "pcib1 memory window"), >>>>> so only the check that should be failing are the checks against = rm_start and >>>>> rm_end. For the memory window, rm_start is always 0, and rm_end = is always >>>>> 0xffffffff, so both the old (0xc00000000 - 0xc00fffff) and new = (0x60000000 - 0x600fffffff) >>>>> ranges are within those bounds. >>=20 >> No: >>=20 >> 0xffffffff >>=20 >> .vs (actual): >>=20 >> 0x600000000 >> 0x6000fffff >>=20 >> Both the actuals are larger then the 0xffffffff figure you list = above. >>=20 >> I've no clue if the 0xffffffff might have its own typos. >>=20 >>>>> I would instead expect to see some other issue later >>>>> on where we fail to allocate a resource for a child BAR, but I = wouldn't expect >>>>> rman_manage_region to fail. >>>>>=20 >>>>> Logging the return value from rman_manage_region would be the = first step I think >>>>> to see which error value it is returning. >>>>=20 >>>> Looking at the code in sys/kern/subr_rman.c for rman_manage_region = I see >>>> the (mostly) return rv related logic only has ENONMEM (explicit = return) and >>>> EBUSY as non-0 possibilities (no other returns). >>>=20 >>> The modern code also has EINVAL via an explicit return. >>>=20 >>>> All the rv references and >>>> all the returns are shown below: >>>>=20 >>>> int rv =3D 0; >>>> . . . >>>=20 >>> The modern code also has here: >>>=20 >>> DPRINTF(("rman_manage_region: <%s> request: start %#jx, end = %#jx\n", >>> rm->rm_descr, start, end)); >>> if (start < rm->rm_start || end > rm->rm_end) >>> return EINVAL; >>>=20 >>> Adding rm->rm_start and rm->rm_end to the message might be >>> appropriate --and would also be enough additional information >>> to know if EINVAL would be returned. >>=20 >> I added such code and built, installed, and tried booting >> a separate kernel. The result was: >>=20 >> rman_manage_region: range 0..0xffffffff : = request: start 0x600000000, end 0x6000fffff >> panic: Failed to add resource to rman >>=20 >> So: >>=20 >> 0 >> vs. start: >> 0x600000000 >>=20 >> and: >>=20 >> 0xffffffff >> vs. end: >> 0x6000fffff >>=20 >> The 0x600000000..0x6000fffff range is not a range of RAM addresses >> [too large for that for the 8 GiByte RPi4B] and, so, is well above >> the 32 bit boundary too. >>=20 >> That is the only line referencing "memory window". A 32 bit initial = range >> for some reason. For reference, the source change in question was: >>=20 >> - DPRINTF(("rman_manage_region: <%s> request: start %#jx, end = %#jx\n", >> - rm->rm_descr, start, end)); >> + DPRINTF(("rman_manage_region: <%s> range %#jx..%#jx : = request: start %#jx, end %#jx\n", >> + rm->rm_descr, rm->rm_start, rm->rm_end, start, end)); >> if (start < rm->rm_start || end > rm->rm_end) >> return EINVAL; >>=20 >> I'll show a larger boot output context at the bottom of the message >> for reference. >>=20 >>>> r =3D int_alloc_resource(M_NOWAIT); >>>> if (r =3D=3D NULL) >>>> return ENOMEM; >>>> . . . >>>> /* Check for any overlap with the current region. */ >>>> if (r->r_start <=3D s->r_end && r->r_end >=3D = s->r_start) { >>>> rv =3D EBUSY; >>>> goto out; >>>> } >>>>=20 >>>> /* Check for any overlap with the next region. */ >>>> t =3D TAILQ_NEXT(s, r_link); >>>> if (t && r->r_start <=3D t->r_end && r->r_end >=3D = t->r_start) { >>>> rv =3D EBUSY; >>>> goto out; >>>> } >>>> . . . >>>> out: >>>> mtx_unlock(rm->rm_mtx); >>>> return rv; >>>>=20 >>>> int_alloc_resource failure would be failure (NULL) from the: >>>>=20 >>>> struct resource_i *r; >>>>=20 >>>> r =3D malloc(sizeof *r, M_RMAN, malloc_flag | M_ZERO); >>>>=20 >>>> (associated with the M_NOWAIT argument). >>>>=20 >>>> The malloc failure would likely go in a very different direction. >>>>=20 >>>>=20 >>>>=20 >>>> Side note: looks like the EBUSY cases leak what r references. >>>=20 >>> Still true in the newer code. >>>=20 >>>>> Probably I should fix pci_host_generic.c to handle translation = properly however. >>>>> I can work on a patch for that. >>>>=20 >>=20 >> For reference: >>=20 >> . . . >> pcib0: mem = 0x7d500000-0x7d50930f irq 80,81 on simplebus2 >> pcib0: parsing FDT for ECAM0: >> pcib0: PCI addr: 0xc0000000, CPU addr: 0x600000000, Size: 0x40000000 >> pcib0: PCI addr: 0x0, CPU addr: 0x0, Size: 0x0 >> pcib0: PCI addr: 0x0, CPU addr: 0x0, Size: 0x0 >> pcib0: PCI addr: 0x0, CPU addr: 0x0, Size: 0x0 >> pcib0: PCI addr: 0x0, CPU addr: 0x0, Size: 0x0 >> pcib0: PCI addr: 0x0, CPU addr: 0x0, Size: 0x0 >> pcib0: PCI addr: 0x0, CPU addr: 0x0, Size: 0x0 >> pcib0: PCI addr: 0x0, CPU addr: 0x0, Size: 0x0 >> pcib0: PCI addr: 0x0, CPU addr: 0x0, Size: 0x0 >> pcib0: PCI addr: 0x0, CPU addr: 0x0, Size: 0x0 >> pcib0: PCI addr: 0x0, CPU addr: 0x0, Size: 0x0 >> pcib0: PCI addr: 0x0, CPU addr: 0x0, Size: 0x0 >> pcib0: PCI addr: 0x0, CPU addr: 0x0, Size: 0x0 >> pcib0: PCI addr: 0x0, CPU addr: 0x0, Size: 0x0 >> pcib0: PCI addr: 0x0, CPU addr: 0x0, Size: 0x0 >> pcib0: PCI addr: 0x0, CPU addr: 0x0, Size: 0x0 >> pcib0: Bus is not cache-coherent >> rman_reserve_resource_bound: request: = [0xfd500000, 0xfd50930f], length 0x9310, flags 100, device pcib0 >> rman_reserve_resource_bound: trying 0x1fff <0xfd500000,0x930f> >> rman_reserve_resource_bound: tried 0x1fff <0xfd500000,0x930f> >> rman_reserve_resource_bound: tried 0x31bfffff <0xfd500000,0x930f> >> rman_reserve_resource_bound: tried 0x33298fff <0xfd500000,0x930f> >> rman_reserve_resource_bound: tried 0x39bf1fff <0xfd500000,0x930f> >> rman_reserve_resource_bound: tried 0x39c02fff <0xfd500000,0x930f> >> rman_reserve_resource_bound: tried 0x39c06fff <0xfd500000,0x930f> >> rman_reserve_resource_bound: tried 0x39c07fff <0xfd500000,0x930f> >> rman_reserve_resource_bound: tried 0x39c08fff <0xfd500000,0x930f> >> rman_reserve_resource_bound: tried 0x39c2afff <0xfd500000,0x930f> >> rman_reserve_resource_bound: tried 0x39c36fff <0xfd500000,0x930f> >> rman_reserve_resource_bound: tried 0x39c37fff <0xfd500000,0x930f> >> rman_reserve_resource_bound: tried 0x3b03ffff <0xfd500000,0x930f> >> rman_reserve_resource_bound: tried 0x3b04ffff <0xfd500000,0x930f> >> rman_reserve_resource_bound: tried 0x3b2fffff <0xfd500000,0x930f> >> rman_reserve_resource_bound: tried 0x3ee61fff <0xfd500000,0x930f> >> rman_reserve_resource_bound: tried 0x3ee63fff <0xfd500000,0x930f> >> rman_reserve_resource_bound: tried 0x3fffffff <0xfd500000,0x930f> >> rman_reserve_resource_bound: tried 0xfbffffff <0xfd500000,0x930f> >> considering [0xfc000000, 0xfd5d1fff] >> truncated region: [0xfd500000, 0xfd50930f]; size 0x9310 (requested = 0x9310) >> candidate region: [0xfd500000, 0xfd50930f], size 0x9310 >> splitting region in three parts: [0xfc000000, 0xfd4fffff]; = [0xfd500000, 0xfd50930f]; [0xfd509310, 0xfd5d1fff] >> rman_manage_region: range 0..0xffffffffffffffff : = request: start 0xc0000000, end 0xffffffff >> pcib0: hardware identifies as revision 0x304. >> pcib0: note: reported link speed is 5.0 GT/s. >> rman_reserve_resource_bound: request: [0x51, 0x51], = length 0x1, flags 0, device pcib0 >> rman_reserve_resource_bound: trying 0 <0x51,0> >> rman_reserve_resource_bound: tried 0 <0x51,0> >> rman_reserve_resource_bound: tried 0x1 <0x51,0> >> rman_reserve_resource_bound: tried 0x2 <0x51,0> >> rman_reserve_resource_bound: tried 0x3 <0x51,0> >> rman_reserve_resource_bound: tried 0x4 <0x51,0> >> rman_reserve_resource_bound: tried 0x5 <0x51,0> >> rman_reserve_resource_bound: tried 0x6 <0x51,0> >> rman_reserve_resource_bound: tried 0x7 <0x51,0> >> rman_reserve_resource_bound: tried 0xc <0x51,0> >> rman_reserve_resource_bound: tried 0xd <0x51,0> >> rman_reserve_resource_bound: tried 0xe <0x51,0> >> rman_reserve_resource_bound: tried 0xf <0x51,0> >> rman_reserve_resource_bound: tried 0x10 <0x51,0> >> rman_reserve_resource_bound: tried 0x11 <0x51,0> >> rman_reserve_resource_bound: tried 0x12 <0x51,0> >> rman_reserve_resource_bound: tried 0x17 <0x51,0> >> rman_reserve_resource_bound: tried 0x18 <0x51,0> >> rman_reserve_resource_bound: tried 0x1a <0x51,0> >> rman_reserve_resource_bound: tried 0x1b <0x51,0> >> rman_reserve_resource_bound: tried 0x22 <0x51,0> >> rman_reserve_resource_bound: tried 0x23 <0x51,0> >> rman_reserve_resource_bound: tried 0x24 <0x51,0> >> rman_reserve_resource_bound: tried 0x25 <0x51,0> >> rman_reserve_resource_bound: tried 0x26 <0x51,0> >> rman_reserve_resource_bound: tried 0x27 <0x51,0> >> rman_reserve_resource_bound: tried 0x28 <0x51,0> >> rman_reserve_resource_bound: tried 0x29 <0x51,0> >> rman_reserve_resource_bound: tried 0x4e <0x51,0> >> rman_reserve_resource_bound: tried 0x4f <0x51,0> >> considering [0x50, 0xffffffffffffffff] >> truncated region: [0x51, 0x51]; size 0x1 (requested 0x1) >> candidate region: [0x51, 0x51], size 0x1 >> splitting region in three parts: [0x50, 0x50]; [0x51, 0x51]; [0x52, = 0xffffffffffffffff] >> pci0: on pcib0 >> rman_manage_region: range 0..0xff : = request: start 0, end 0xff >> rman_reserve_resource_bound: request: [0, = 0], length 0x1, flags 0, device pci0 >> rman_reserve_resource_bound: trying 0xff <0,0> >> considering [0, 0xff] >> truncated region: [0, 0]; size 0x1 (requested 0x1) >> candidate region: [0, 0], size 0x1 >> allocating from the beginning >> pci0: domain=3D0, physical bus=3D0 >> found-> vendor=3D0x14e4, dev=3D0x2711, revid=3D0x00 >> domain=3D0, bus=3D0, slot=3D0, func=3D0 >> class=3D06-04-00, hdrtype=3D0x01, mfdev=3D0 >> cmdreg=3D0x0000, statreg=3D0x0010, cachelnsz=3D0 (dwords) >> lattimer=3D0x00 (0 ns), mingnt=3D0x00 (0 ns), maxlat=3D0x00 (0 = ns) >> intpin=3Da, irq=3D0 >> powerspec 3 supports D0 D3 current D0 >> secbus=3D1, subbus=3D1 >> rman_reserve_resource_bound: request: = [0x1, 0x1], length 0x1, flags 0, device (null) >> rman_reserve_resource_bound: trying 0 <0x1,0> >> rman_reserve_resource_bound: tried 0 <0x1,0> >> considering [0x1, 0xff] >> truncated region: [0x1, 0x1]; size 0x1 (requested 0x1) >> candidate region: [0x1, 0x1], size 0x1 >> allocating from the beginning >> pcib1: irq 91 at device 0.0 on pci0 >> rman_manage_region: range 0..0xff : request: = start 0x1, end 0x1 >> pcib0: rman_reserve_resource: start=3D0xc0000000, end=3D0xc00fffff, = count=3D0x100000 >> rman_reserve_resource_bound: request: [0xc0000000, = 0xc00fffff], length 0x100000, flags 102, device pcib1 >> rman_reserve_resource_bound: trying 0xffffffff <0xc0000000,0xfffff> >> considering [0xc0000000, 0xffffffff] >> truncated region: [0xc0000000, 0xc00fffff]; size 0x100000 (requested = 0x100000) >> candidate region: [0xc0000000, 0xc00fffff], size 0x100000 >> allocating from the beginning >> rman_manage_region: range 0..0xffffffff : = request: start 0x600000000, end 0x6000fffff >> panic: Failed to add resource to rman >> cpuid =3D 0 >> time =3D 1 >> KDB: stack backtrace: >> db_trace_self() at db_trace_self >> db_trace_self_wrapper() at db_trace_self_wrapper+0x38 >> vpanic() at vpanic+0x1a4 >> panic() at panic+0x48 >> pcib_add_window_resources() at pcib_add_window_resources+0xf4 >> pcib_alloc_window() at pcib_alloc_window+0xfc >> pcib_attach_common() at pcib_attach_common+0xa18 >> pcib_attach() at pcib_attach+0x14 >> device_attach() at device_attach+0x3fc >> device_probe_and_attach() at device_probe_and_attach+0x80 >> bus_generic_attach() at bus_generic_attach+0x1c >> pci_attach() at pci_attach+0xec >> device_attach() at device_attach+0x3fc >> device_probe_and_attach() at device_probe_and_attach+0x80 >> bus_generic_attach() at bus_generic_attach+0x1c >> device_attach() at device_attach+0x3fc >> device_probe_and_attach() at device_probe_and_attach+0x80 >> bus_generic_new_pass() at bus_generic_new_pass+0x100 >> bus_generic_new_pass() at bus_generic_new_pass+0xb0 >> bus_generic_new_pass() at bus_generic_new_pass+0xb0 >> bus_generic_new_pass() at bus_generic_new_pass+0xb0 >> bus_set_pass() at bus_set_pass+0x50 >> mi_startup() at mi_startup+0x1e0 >> virtdone() at virtdone+0x68 >> KDB: enter: panic >> [ thread pid 0 tid 100000 ] >> Stopped at kdb_enter+0x4c: str xzr, [x19, #1280] >> db>=20 >>=20 >=20 > FYI: >=20 > bcm2711-peripherals.pdf "Figure 1. BCM2711 Address Maps" > documents the PCIe address space range as (their "_" > notation): >=20 > 0x6_0000_0000..0x7_FFFF_FFFF >=20 > for both of the address maps: >=20 > 'Full 35-bit Address Map' > 'ARM view of the Address Map in "Low Peripheral" mode' >=20 > The Low Peripheral mode is what is in use. >=20 > So the 0x6_0000_0000..0x6_000f_ffff range looks > to be a valid ARM system address space for the > PCIe to map to. It looks to me like in sys/dev/pci/pci_pci.c the: static void pcib_probe_windows(struct pcib_softc *sc) { . . . pcib_alloc_window(sc, &sc->mem, SYS_RES_MEMORY, 0, 0xffffffff); . . . is just inappropriately restrictive about where in the system address space a PCIe can validly be mapped to on the high end. That, in turn, leads to the rejection on the RPi4B now that the range use is checked. More: Even the size (0xffffffff-0)+1 is smaller than the size of the range that the BCM2711 documentation indicates as reserved for PCIe (not that the fdt indicates to use the upper half of that range at all): (0x7_FFFF_FFFF - 0x6_0000_0000)+1 =3D=3D 0x1_FFFF_FFFF + 1 (I'm not so sure that this matters for the RPi4B. It is not a great example of the generality of PCIe uses. But might there be contexts where it does matter?) The lower bound being 0 allows for the actual low bound. But I wonder if use of the fdt (or ACPI) information should be possible to then indicate a actual low bound to check against instead. (A smaller high bound may be reasonable, at least for a RPi4B like context.) =3D=3D=3D Mark Millard marklmi at yahoo.com