From nobody Thu Apr 21 19:02:35 2022 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 2AECF1992CD4; Thu, 21 Apr 2022 19:02:36 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Kkn3b5l3kz3C0F; Thu, 21 Apr 2022 19:02:35 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1650567755; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=CMpmZzu3+ncmFZvfxv6j8+UKZ4zsjC6FSrniz+cOdEg=; b=Lg+F/p5u5ScdEbsnAYuQ/+B11D+RTDxtttpM3FS/1C3ZvPRhaA+RRvL6QVrOcYnFrUigFS ElLr+to04F44xviQb1GA4f3ehwLzR1oxTMD+P+Iie55gQ91/7Tj92VojF59n3AzEFJc3Uc Zp5zLFj7ZG6276k2JgOCrHr7/iVKDK+P78fgdxF7mqUK+Ggfh4vK1d7CDFgt4FTkqPsApP 1A5VGBLMFpb2dK0zZVYcGa4+OgQl/ua7mHauHR30VPVNbsTz7wqsb92XewG+gRzLksKS8E Ulljd/guppZO9AsoZuuRfEk9WEcWF16vZjVQBVMRKQoo6PNTJjm735vstquRyw== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 92619679F; Thu, 21 Apr 2022 19:02:35 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 23LJ2Zht035845; Thu, 21 Apr 2022 19:02:35 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 23LJ2ZJs035844; Thu, 21 Apr 2022 19:02:35 GMT (envelope-from git) Date: Thu, 21 Apr 2022 19:02:35 GMT Message-Id: <202204211902.23LJ2ZJs035844@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: John Baldwin Subject: git: 10fe9a1fb4df - main - busdma_bounce: Make the map waiting list per-bounce-zone. List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-all@freebsd.org X-BeenThere: dev-commits-src-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: jhb X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 10fe9a1fb4df5a55e3dae6017034f48a4ff02045 Auto-Submitted: auto-generated ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1650567755; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=CMpmZzu3+ncmFZvfxv6j8+UKZ4zsjC6FSrniz+cOdEg=; b=ptk7DW2qybKHW6saXk8Fgit1xxQFbtIoNG6bf+6dzUfn5q1N19a9pVgMN0VzvuVYS6yauQ /AChwPLTXHNQCe7bLCtpkkDgykE/0FmWDPaWwYu0lM2CiTLWZ8xeZiPo/Y82eRMutBpJYp zx0yPwwi59vlQ644gGB8jhFJOf2bFJpLB18yUpgsOyPo6A58XvNB43YVsgD/KouWDal40Y LP6DYwclbw8mKRFMs9CerlW/d+0QUPbeq4RD3pMuecXpvH1L0fKSPDLAq6helx8GaPH2c/ 09lS+DWi19IONaY7n0b89RWsq7nUjAGrnUWm7IgfHZWLfOCvlU9Yi6rMUzoNVw== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1650567755; a=rsa-sha256; cv=none; b=WANvW5gVzjuEhPex6Lg9aLqIUKC/cZRZGS+1p0+9snY4R66saoL39SP7QMG2HIO75leI6g ibrLanSdGYunidWFT9rH0T+NtbFj8EDp5IeSnHQcdX625YBhzfFg0Uo2BaHmObR06kWnql 9dxbjR+zLt2oz9t0c3zPYT92Y8fp93eRsf0KjRIIkcVLlEEognJIegsRs7yCx5Vnp27Ids jJMn4zoukvP8kH637NTiCoHL824rorZNazjlDn613ohDveHnZemaKQlg91STKVhXPeaGBQ HlhtzrA99wbK8ARihfTC5Wxxk3AwEc6KhrWmi860yHaDGA6DeIIMQsU2in4Dow== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N The branch main has been updated by jhb: URL: https://cgit.FreeBSD.org/src/commit/?id=10fe9a1fb4df5a55e3dae6017034f48a4ff02045 commit 10fe9a1fb4df5a55e3dae6017034f48a4ff02045 Author: John Baldwin AuthorDate: 2022-04-21 17:41:09 +0000 Commit: John Baldwin CommitDate: 2022-04-21 17:41:09 +0000 busdma_bounce: Make the map waiting list per-bounce-zone. When pages are freed to a bounce zone, only maps waiting for pages for that zone can make forward progress. If a map for a different bounce zone is at the head of the global list, then requests that could otherwise make forward progress will be stalled waiting on the other bounce zone. If bounce zones shared bounce pages then a global list would still make sense to prevent "later" requests from starving an earlier request but that is not a concern with per-zone bounce page pools. Reviewed by: imp Sponsored by: Netflix Differential Revision: https://reviews.freebsd.org/D34966 --- sys/kern/subr_busdma_bounce.c | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/sys/kern/subr_busdma_bounce.c b/sys/kern/subr_busdma_bounce.c index 2c5cc1ea83a2..586998e01673 100644 --- a/sys/kern/subr_busdma_bounce.c +++ b/sys/kern/subr_busdma_bounce.c @@ -65,6 +65,7 @@ struct bounce_page { struct bounce_zone { STAILQ_ENTRY(bounce_zone) links; STAILQ_HEAD(, bounce_page) bounce_page_list; + STAILQ_HEAD(, bus_dmamap) bounce_map_waitinglist; int total_bpages; int free_bpages; int reserved_bpages; @@ -88,7 +89,6 @@ static int total_bpages; static int busdma_zonecount; static STAILQ_HEAD(, bounce_zone) bounce_zone_list; -static STAILQ_HEAD(, bus_dmamap) bounce_map_waitinglist; static STAILQ_HEAD(, bus_dmamap) bounce_map_callbacklist; static MALLOC_DEFINE(M_BOUNCE, "bounce", "busdma bounce pages"); @@ -103,6 +103,7 @@ static int reserve_bounce_pages(bus_dma_tag_t dmat, bus_dmamap_t map, static int _bus_dmamap_reserve_pages(bus_dma_tag_t dmat, bus_dmamap_t map, int flags) { + struct bounce_zone *bz; /* Reserve Necessary Bounce Pages */ mtx_lock(&bounce_lock); @@ -115,7 +116,9 @@ _bus_dmamap_reserve_pages(bus_dma_tag_t dmat, bus_dmamap_t map, int flags) } else { if (reserve_bounce_pages(dmat, map, 1) != 0) { /* Queue us for resources */ - STAILQ_INSERT_TAIL(&bounce_map_waitinglist, map, links); + bz = dmat->bounce_zone; + STAILQ_INSERT_TAIL(&bz->bounce_map_waitinglist, map, + links); mtx_unlock(&bounce_lock); return (EINPROGRESS); } @@ -131,7 +134,6 @@ init_bounce_pages(void *dummy __unused) total_bpages = 0; STAILQ_INIT(&bounce_zone_list); - STAILQ_INIT(&bounce_map_waitinglist); STAILQ_INIT(&bounce_map_callbacklist); mtx_init(&bounce_lock, "bounce pages lock", NULL, MTX_DEF); } @@ -174,6 +176,7 @@ alloc_bounce_zone(bus_dma_tag_t dmat) return (ENOMEM); STAILQ_INIT(&bz->bounce_page_list); + STAILQ_INIT(&bz->bounce_map_waitinglist); bz->free_bpages = 0; bz->reserved_bpages = 0; bz->active_bpages = 0; @@ -403,9 +406,9 @@ free_bounce_page(bus_dma_tag_t dmat, struct bounce_page *bpage) STAILQ_INSERT_HEAD(&bz->bounce_page_list, bpage, links); bz->free_bpages++; bz->active_bpages--; - if ((map = STAILQ_FIRST(&bounce_map_waitinglist)) != NULL) { + if ((map = STAILQ_FIRST(&bz->bounce_map_waitinglist)) != NULL) { if (reserve_bounce_pages(map->dmat, map, 1) == 0) { - STAILQ_REMOVE_HEAD(&bounce_map_waitinglist, links); + STAILQ_REMOVE_HEAD(&bz->bounce_map_waitinglist, links); STAILQ_INSERT_TAIL(&bounce_map_callbacklist, map, links); bz->total_deferred++;