From nobody Mon Jan 22 06:55:58 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TJLbf0T1cz57FPS for ; Mon, 22 Jan 2024 06:56:14 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic308-8.consmr.mail.gq1.yahoo.com (sonic308-8.consmr.mail.gq1.yahoo.com [98.137.68.32]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4TJLbc4xPYz42rf for ; Mon, 22 Jan 2024 06:56:12 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=li0k1ghU; dmarc=pass (policy=reject) header.from=yahoo.com; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.68.32 as permitted sender) smtp.mailfrom=marklmi@yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1705906570; bh=oOcWTFBGzs8eNkLyPPmb5vsartPyMqM/sTeS7V9VAu0=; h=From:Subject:Date:To:References:From:Subject:Reply-To; b=li0k1ghUTrYKIcRwG1nTr3NcaP8YxhbouhLPCGUjkzq4sa3htXM6Ub5/IG1xRs6gKNRP8r4narCQpZrUj16BOTJ1XVDSGlDic7VDe2YDFP4oTjLxlfqNDgkydET7xWbSonpsTPXtJxr64vf4Q6DWCGpgQll/0oDAC/3jd7MnnQkzHYpODbzT61V4BoakvbE4Xf+fWnRttkUx9lWWuTyw853xJLp/FtYJO3u++I1dlWGJys+6awU8vmIFQYYtmAi56zo4NUK9nwC0/INNADL7RFa4guQS35Nuwu/YvNuhBKhSCfVCvVas8t19WOdYcT3EM4Scyjblw1a2qJ0MLlAF5w== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1705906570; bh=KCariv8OCeZv4RGPEee2ng0cfPemADzV/tK4S6l9PO2=; h=X-Sonic-MF:From:Subject:Date:To:From:Subject; b=KyyRp/xLo+ilK07ha9a2+OfPqzFU5ejOu9YY7mLZtprZokwCak3wKbNVFrvUKTOsuYa+Vugy5yglupgspKpFUbUeopwkF3U9gjn9PpXhKWKgYTowUoJzl2qHdkqoLiSv7i5SdN78jz3RijnaxpLP7mcx93ir7oKXYar878CPiWh4kWMYyclFaCRj0GMnX6TJTkorwKBGbX2y2OPle2kQof1qULpgtr3soKQXyIN1diPKV51X3jhN0CcvkCXgYtnzrNYB7/8OGEfnIyPyXJoWSF0rh7KM5Z8KZC489vxi/0vpu1hn80VsC6OWgH3pBg+uHMvMI4OFXDJutJF4VdHx+A== X-YMail-OSG: WzAeECoVM1m.1ykm98snWww.nEaFk.52ObeE0aQlaAjSorZ2.c08KfWe2MRk63f PcZl8kHOfKa0APtZ9LXlPbF69VFntoj1B2k_eKBporB2NsLU30xD.46rfNwwDYbTA4SCtIs8wqpW nkGHu3F7tFWOzQ4oVQ2bYGW2GVRAF9pNuNHAmBdmHWz35Ejj82roNzyDE3zDSX1owpJApzm0QU4D C4HegWSBfoDyo2hUhb7J55UfZUrZ2sp1xsUYp0kGwe2OsBZkcDkCFxiLOKeEfC8yEPFNqwGqOaJi tibqMdslvfiNQORqruRWtmnWu5oOF.hoaEkrh1KgVJ_4QO_3fTiyhsGSXBrBwOSBKrIn.R48w7Na Ys6hjxusEmVtLFxFd2tRO9FwGyD3TqOcPhhFPzYRiiuXulqDec8ovj23Y6hQngXcp0na44o3OR0p phjH6Urbvh6s3lZ.ZgI.lWr4dLdrGdN2ZpVVaUxbWj299nFTnTOmXeBgfX4H_8htaDb_PaTZOP7n 7xiCq0UuKLSqFs3gigqI9pcWkEDMHxjJiVHiyhGCq5wVISBs7KmbE.OjHd8tbOeIvqiqK.0yxiS4 ys91rwfM.JEFLoRRaBqePmN0loQd9E0wzolfXjaUPRq0e1JSrRsiAP27YSBN4zihsHEPAMLT7fQq odTxXWNbD8309c7g2vzfCKiM0s4Y4hXG_QxyTgNuW8NQVOqcD.GRJ2qaLwW3S.xRzwNdM8ekZuRc BgrHM6dOJP3QID80EOhm.ijRUERUKZW0juRmZW0VMN7FjEclUBqhR1wR1eVOoXXHPEo5AOeRYRKc gQUslJdOA1uKZJiRDR4ACjF8gepIC1w38Ds3ZYOM7ujMfnsUk_yl2ZUfswu2zpxj7LT8Diw3RvaX 3_CqCHwLkAvnPZoBTlO1dtG4NbXXqnBmlU_SaP4.hrRk0LvgQDQr33DYzkKoDfdZXaZTuNNOfg_d _qryu8stVhvRlKQ4BQ4rv5o6_v4uDUr_nm1EYBrHFER_Yyq4J8DvT84pZJdvDMgLebWdu0G8F5yK MFdUncynSG7EA.ZQaFiM9e.1XP_aHQMaxC9IwHaECM_IyfIZuIEhPqb2MCKqOCg2kjYE.CXMGMiE 8mh8jRzihKqc2gdsXcSpvLDUG_VvcybankBkPu8aewtFQeOXT36GfOEimivvytCnjSlHd2mzOz53 oZhqSiQIvOQrVMlpsnh.oXOsOHCuK44JcDpP8.xErubzgk4rEp0otn4l.lOYjJl4ehrV84qLF_6H uASrxgew4u1JQT5FoqFOz4XXlf5PZ2KFWUuSnKpkRktpx8YEhKFCsxDSXO0obyh6FxH6aVFIgzWk DQ3QBWc_9cS5tFTjG66rw_v3Tlhrc.q54GFr8NMz_c6l.LY_2gH0VVvC8fuhkjYyC_liv_yb65cz jcV_CTGwPFyv67.efC3XTpv397lcnw_0yOH41Wg3FQZ9pe8wtmDS31Xo6C0zjteLxycrEjapFnkP m8I6.Bra1xpmQfAMbWNzfwRigd7PVw6tcA9qCVk3GnoRTn3brzhH9x2JU5k_NA4JDFLzTJvF_rTI k2x7mF4ytgow62v8yRwelxs3Fv7A5zCbWbJmxeC6eu3Jk1LnWX8u_YGO8IYNBuyDuJ6hWWB3.z1J X5rx_Q0oTpD24w1nSsIwxOWepuHOIyK5dZmG2XRTdfnzvoDicKdDk26zfjeNX33xcdoKlds1bm0U 6PTJP33VOP5r_5qbz41zpYO26X5CMF0szrIkFy4YU7I_BCfVGHWn2qIsU2l3UOPiLVsGth6DBTwS RDrjVIKDzT9H__th7CqMUWE67MrCxilaGHZ96XPisNH.7.fk_42Hbtgeyp2w2.B6h9g_gigB4C1R s.5kmq.uVasemIgVq4.ZkV9zNn4GTHT15tPTGLWsc6PFPcPdsj3PFmll8a552MQo87tmsI9w2V3F OyCYSoYNbo2rMrBx0.euoDvN52Pnr8zgjO1et3xFoGWkowCI4_mie1oZojH1T9_oqt_khlBxhFob .EsSbQmG._WiW6aMUHjsVYQjNz4FKba8Uu2KPArhk0mrfbUxu4hEugFaLi0nU3zNQxhv3EtY3LW6 fZ9qtbqI.cw9rg7oovaxSkpilrLrzEWopr32WQgZbvZrvzrZVbRYsbko0xY2HOfzNXQ1yOAJmjBx dZBZLAzxp9bs9HfXNm.Kni1JnzVy0x0o8YyDglNlsXhnnLq8O64ikJ9uuYc3cg0r9Lwnt6A1hkT8 VxFFK8nhlmG3zhxXaxn4KD_Dmk3YrFjROdE9aSwyiKKfkWwt0D24.gk3oqg-- X-Sonic-MF: X-Sonic-ID: e2b1e58d-b303-4325-8d6a-c90c6adc2c6c Received: from sonic.gate.mail.ne1.yahoo.com by sonic308.consmr.mail.gq1.yahoo.com with HTTP; Mon, 22 Jan 2024 06:56:10 +0000 Received: by hermes--production-gq1-78d49cd6df-tswkb (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 2246da0ae1f18cc38d754507f535c748; Mon, 22 Jan 2024 06:56:09 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.300.61.1.2\)) Subject: SDHCI_QUIRK_BROKEN_SDMA_BOUNDARY, maxphys, SDHCI_BLKSZ_SDMA_BNDRY_512K, and alloc_bounce_zone behavior [RPi5 example] Message-Id: Date: Sun, 21 Jan 2024 22:55:58 -0800 To: Current FreeBSD X-Mailer: Apple Mail (2.3774.300.61.1.2) References: X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.50 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; MV_CASE(0.50)[]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; MIME_GOOD(-0.10)[text/plain]; TO_DN_ALL(0.00)[]; RCVD_TLS_LAST(0.00)[]; FREEMAIL_FROM(0.00)[yahoo.com]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; ARC_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; FROM_HAS_DN(0.00)[]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; RCVD_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.68.32:from]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RCVD_IN_DNSWL_NONE(0.00)[98.137.68.32:from] X-Rspamd-Queue-Id: 4TJLbc4xPYz42rf I had previously reported on the freebsd-arm list that on the RPi5 used via the EDK2 draft I was seeing: # sysctl hw.busdma hw.busdma.zone0.total_deferred_time: 0 0 hw.busdma.zone0.domain: 0 hw.busdma.zone0.alignment: 524288 hw.busdma.zone0.lowaddr: 0xffffffff hw.busdma.zone0.total_deferred: 0 hw.busdma.zone0.total_bounced: 12018773 hw.busdma.zone0.active_bpages: 12 hw.busdma.zone0.reserved_bpages: 0 hw.busdma.zone0.free_bpages: 1227 hw.busdma.zone0.total_bpages: 1239 hw.busdma.total_bpages: 1239 Note the large alignment. It turns out that the first alloc_bounce_zone call on the RPi5 is for: if (!(slot->quirks & SDHCI_QUIRK_BROKEN_SDMA_BOUNDARY)) { if (maxphys <=3D 1024 * 4) slot->sdma_boundary =3D = SDHCI_BLKSZ_SDMA_BNDRY_4K; else if (maxphys <=3D 1024 * 8) slot->sdma_boundary =3D = SDHCI_BLKSZ_SDMA_BNDRY_8K; else if (maxphys <=3D 1024 * 16) slot->sdma_boundary =3D = SDHCI_BLKSZ_SDMA_BNDRY_16K; else if (maxphys <=3D 1024 * 32) slot->sdma_boundary =3D = SDHCI_BLKSZ_SDMA_BNDRY_32K; else if (maxphys <=3D 1024 * 64) slot->sdma_boundary =3D = SDHCI_BLKSZ_SDMA_BNDRY_64K; else if (maxphys <=3D 1024 * 128) slot->sdma_boundary =3D = SDHCI_BLKSZ_SDMA_BNDRY_128K; else if (maxphys <=3D 1024 * 256) slot->sdma_boundary =3D = SDHCI_BLKSZ_SDMA_BNDRY_256K; else slot->sdma_boundary =3D = SDHCI_BLKSZ_SDMA_BNDRY_512K; } slot->sdma_bbufsz =3D = SDHCI_SDMA_BNDRY_TO_BBUFSZ(slot->sdma_boundary); /* * Allocate the DMA tag for an SDMA bounce buffer. * Note that the SDHCI specification doesn't state any alignment * constraint for the SDMA system address. However, controllers * typically ignore the SDMA boundary bits in SDHCI_DMA_ADDRESS = when * forming the actual address of data, requiring the SDMA buffer = to * be aligned to the SDMA boundary. */ err =3D bus_dma_tag_create(bus_get_dma_tag(slot->bus), = slot->sdma_bbufsz, 0, BUS_SPACE_MAXADDR_32BIT, BUS_SPACE_MAXADDR, NULL, NULL, slot->sdma_bbufsz, 1, slot->sdma_bbufsz, BUS_DMA_ALLOCNOW, NULL, NULL, &slot->dmatag); That gives the alignment 524288 and lowaddr 0xffffffff on the RPi5 used via the EDK2 draft. That gives only 8192 aligment positions that fit in the 32 bit address space lowaddr spans. alloc_bounce_zone does: . . . /* Check to see if we already have a suitable zone */ STAILQ_FOREACH(bz, &bounce_zone_list, links) { if ((dmat_alignment(dmat) <=3D bz->alignment) && #ifdef dmat_domain =20 dmat_domain(dmat) =3D=3D bz->domain && #endif (dmat_lowaddr(dmat) >=3D bz->lowaddr)) { dmat->bounce_zone =3D bz; return (0); } =20 } . . . bz->alignment =3D MAX(dmat_alignment(dmat), PAGE_SIZE); . . . so later calls end up with: dmat_alignment(dmat) <=3D bz->alignment // actually < 524288 and: dmat_lowaddr(dmat) >=3D bz->lowaddr // actually > 0xffffffffu So everything ends up using: hw.busdma.zone0.alignment: 524288 hw.busdma.zone0.lowaddr: 0xffffffff and no other busdma zone is created for the RPi5 EDK2 context. One could imaging a smaller lowaddr being in the first alloc_bounce_zone call --and/or a larger alignment. The code is sensitive to the ordering that the value pairs happen to occur in and just a reversed order could give very different results, for example. I would expect that avoiding everything sharing a small lowaddr value or huge alignment value (or a combination of both) would be appropriate. In other words, I'm suggesting that alloc_bounce_zone probably should be adjusted. I'll warn that there is another oddity that many calls to alloc_bounce_zone end up with dmat_alignment(dmat) being < PAGE_SIZE, tiny even. But the bz->alignment was forced to a PAGE_SIZE as a minimum value in the first call. The comparison is not based on a common value-standard for the 2 sides. =3D=3D=3D Mark Millard marklmi at yahoo.com