From nobody Thu Apr 21 19:02:34 2022 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id C438F1992E80; Thu, 21 Apr 2022 19:02:34 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Kkn3Z4FHCz3Btw; Thu, 21 Apr 2022 19:02:34 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1650567754; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=ntUJ1tqwM8DIv0fltyThSBkxW1bl5mvxfRk9d8zMXFk=; b=d+2pkGhBmef3SsUZZWAXFbWgUNBZRuRZ5QNV5LHiGVNxaKUu8FqJid28IycIeAl91CzsjQ 7TupNfvDMZv0FSGUNZwi5kY7KQeWUJiskechiGKlMg6C2nUYDZgcihFxu6FxyO7L+weyMi 1LMpecqOiGExWDUDNqm1krCrOlyiAoVnBElTN5uDYIvkh8I7TGbEwxEvMKrYO0fDV0dClm MdlZDTwgI8D/j15v6vEX/FqXA47+ENpuk4YTxEL4W28bMnRcue7pulFg6Bt+vFqNF8CgVu WE+kLVJH2yc+0AJjkjU97CSBwrFEVTQ7JewHJCK1KL4QXkB/8BkCACUES5US8Q== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 65DC066C8; Thu, 21 Apr 2022 19:02:34 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 23LJ2YPS035821; Thu, 21 Apr 2022 19:02:34 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 23LJ2Y86035820; Thu, 21 Apr 2022 19:02:34 GMT (envelope-from git) Date: Thu, 21 Apr 2022 19:02:34 GMT Message-Id: <202204211902.23LJ2Y86035820@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: John Baldwin Subject: git: d11f5d4762c5 - main - busdma_bounce: Use a simple kproc to invoke deferred requests. List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-all@freebsd.org X-BeenThere: dev-commits-src-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: jhb X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: d11f5d4762c53e6a26f005386ddfa9e67120c15c Auto-Submitted: auto-generated ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1650567754; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=ntUJ1tqwM8DIv0fltyThSBkxW1bl5mvxfRk9d8zMXFk=; b=yFryieVywwaT/dS9DZGiQrudv1kZwzbkTJINmhon6PwEBDqkrV0Rh/Y6fUOUlWJ9Wmr7KZ XYzMe2HOLuL8g3pyVjrnMSxw7aseQOtDUmcv6tCfwPUyjoL+DbwsjeOg5IlQuk3ArVpDPO 7KaAj11JJnryAouHsWoDcnSZQVntiYhME1cvalQflhbkD+ZPEn5e/Dwp+ZlvXPAaQZsJ+H Hg/FOggQy2nEy8ZJI7hvyINBQx4V64s10/2cywRhW3zJjMbKumGay387RtrRLWDTCwwXzn nrF9KwDhxd0aVaGvndQQS0d3FWT6vXU5CQfoPbZKjL0SfNGaoipD4cQ57OxYtA== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1650567754; a=rsa-sha256; cv=none; b=ytH/6QzygheWLs4QuwhGcyL8tyy6i7JaBu0XVrVP6/y9jHMBL7Y1wqdfK8PEdh/cZORuyg 2DTwzNFhukBL6DsqAvLX/qYm1iBEFr6dpTqW3+CfrRn7NLJ2/cgD3JnJz/H+4M3Qx2+9Ys GHf0si8snkmmKnds8zWuNz+sJ/CZxP4y9yqk0o2V119Kmh2k7hrafw85zHW9Wl345MVwPq 93/eYkNrYK5spv3cQOU8wkqM4r++SkEl8BLLIhwL4w0ZxU4GDU+Ha3xeZiBGxws33vq1KN QRw5zA2a9ZyDmdUYaTu8xlISPhMPx7uKKQhsZjYJSNH/ZVy53CcJgZPaT2WT7w== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N The branch main has been updated by jhb: URL: https://cgit.FreeBSD.org/src/commit/?id=d11f5d4762c53e6a26f005386ddfa9e67120c15c commit d11f5d4762c53e6a26f005386ddfa9e67120c15c Author: John Baldwin AuthorDate: 2022-04-21 17:40:35 +0000 Commit: John Baldwin CommitDate: 2022-04-21 17:40:35 +0000 busdma_bounce: Use a simple kproc to invoke deferred requests. Rather than using a software interrupt with a single handler, just create a dedicated kernel process woken up with a simple wakeup(). Reviewed by: imp Sponsored by: Netflix Differential Revision: https://reviews.freebsd.org/D34965 --- sys/kern/subr_busdma_bounce.c | 67 +++++++++++++++++++++++++------------------ 1 file changed, 39 insertions(+), 28 deletions(-) diff --git a/sys/kern/subr_busdma_bounce.c b/sys/kern/subr_busdma_bounce.c index e7a387ffd71d..2c5cc1ea83a2 100644 --- a/sys/kern/subr_busdma_bounce.c +++ b/sys/kern/subr_busdma_bounce.c @@ -45,6 +45,9 @@ * - dmat_lockarg() */ +#include +#include + struct bounce_page { vm_offset_t vaddr; /* kva of bounce buffer */ bus_addr_t busaddr; /* Physical address */ @@ -87,13 +90,13 @@ static int busdma_zonecount; static STAILQ_HEAD(, bounce_zone) bounce_zone_list; static STAILQ_HEAD(, bus_dmamap) bounce_map_waitinglist; static STAILQ_HEAD(, bus_dmamap) bounce_map_callbacklist; -static void *busdma_ih; static MALLOC_DEFINE(M_BOUNCE, "bounce", "busdma bounce pages"); SYSCTL_INT(_hw_busdma, OID_AUTO, total_bpages, CTLFLAG_RD, &total_bpages, 0, "Total bounce pages"); +static void busdma_thread(void *); static int reserve_bounce_pages(bus_dma_tag_t dmat, bus_dmamap_t map, int commit); @@ -152,6 +155,7 @@ static int alloc_bounce_zone(bus_dma_tag_t dmat) { struct bounce_zone *bz; + bool start_thread; /* Check to see if we already have a suitable zone */ STAILQ_FOREACH(bz, &bounce_zone_list, links) { @@ -183,6 +187,7 @@ alloc_bounce_zone(bus_dma_tag_t dmat) busdma_zonecount++; snprintf(bz->lowaddrid, sizeof(bz->lowaddrid), "%#jx", (uintmax_t)bz->lowaddr); + start_thread = STAILQ_EMPTY(&bounce_zone_list); STAILQ_INSERT_TAIL(&bounce_zone_list, bz, links); dmat->bounce_zone = bz; @@ -232,6 +237,11 @@ alloc_bounce_zone(bus_dma_tag_t dmat) "memory domain"); #endif + if (start_thread) { + if (kproc_create(busdma_thread, NULL, NULL, 0, 0, "busdma") != + 0) + printf("failed to create busdma thread"); + } return (0); } @@ -373,7 +383,7 @@ free_bounce_page(bus_dma_tag_t dmat, struct bounce_page *bpage) { struct bus_dmamap *map; struct bounce_zone *bz; - bool schedule_swi; + bool schedule_thread; bz = dmat->bounce_zone; bpage->datavaddr = 0; @@ -388,7 +398,7 @@ free_bounce_page(bus_dma_tag_t dmat, struct bounce_page *bpage) bpage->busaddr &= ~PAGE_MASK; } - schedule_swi = false; + schedule_thread = false; mtx_lock(&bounce_lock); STAILQ_INSERT_HEAD(&bz->bounce_page_list, bpage, links); bz->free_bpages++; @@ -399,40 +409,41 @@ free_bounce_page(bus_dma_tag_t dmat, struct bounce_page *bpage) STAILQ_INSERT_TAIL(&bounce_map_callbacklist, map, links); bz->total_deferred++; - schedule_swi = true; + schedule_thread = true; } } mtx_unlock(&bounce_lock); - if (schedule_swi) - swi_sched(busdma_ih, 0); + if (schedule_thread) + wakeup(&bounce_map_callbacklist); } static void -busdma_swi(void *dummy __unused) +busdma_thread(void *dummy __unused) { + STAILQ_HEAD(, bus_dmamap) callbacklist; bus_dma_tag_t dmat; - struct bus_dmamap *map; + struct bus_dmamap *map, *nmap; - mtx_lock(&bounce_lock); - while ((map = STAILQ_FIRST(&bounce_map_callbacklist)) != NULL) { - STAILQ_REMOVE_HEAD(&bounce_map_callbacklist, links); - mtx_unlock(&bounce_lock); - dmat = map->dmat; - dmat_lockfunc(dmat)(dmat_lockfuncarg(dmat), BUS_DMA_LOCK); - bus_dmamap_load_mem(map->dmat, map, &map->mem, map->callback, - map->callback_arg, BUS_DMA_WAITOK); - dmat_lockfunc(dmat)(dmat_lockfuncarg(dmat), BUS_DMA_UNLOCK); + thread_lock(curthread); + sched_prio(curthread, PI_SWI(SWI_BUSDMA)); + thread_unlock(curthread); + for (;;) { mtx_lock(&bounce_lock); - } - mtx_unlock(&bounce_lock); -} + while (STAILQ_EMPTY(&bounce_map_callbacklist)) + mtx_sleep(&bounce_map_callbacklist, &bounce_lock, 0, + "-", 0); + STAILQ_INIT(&callbacklist); + STAILQ_CONCAT(&callbacklist, &bounce_map_callbacklist); + mtx_unlock(&bounce_lock); -static void -start_busdma_swi(void *dummy __unused) -{ - if (swi_add(NULL, "busdma", busdma_swi, NULL, SWI_BUSDMA, INTR_MPSAFE, - &busdma_ih)) - panic("died while creating busdma swi ithread"); + STAILQ_FOREACH_SAFE(map, &callbacklist, links, nmap) { + dmat = map->dmat; + dmat_lockfunc(dmat)(dmat_lockfuncarg(dmat), + BUS_DMA_LOCK); + bus_dmamap_load_mem(map->dmat, map, &map->mem, + map->callback, map->callback_arg, BUS_DMA_WAITOK); + dmat_lockfunc(dmat)(dmat_lockfuncarg(dmat), + BUS_DMA_UNLOCK); + } + } } -SYSINIT(start_busdma_swi, SI_SUB_SOFTINTR, SI_ORDER_ANY, start_busdma_swi, - NULL);