From nobody Tue May 30 19:30:07 2023 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4QW2Xv6WtXz4Y256; Tue, 30 May 2023 19:30:07 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4QW2Xv614jz48vG; Tue, 30 May 2023 19:30:07 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1685475007; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=2uQsYCyOprB9lncmssuCpU619HEKxhlNXL0lD5H84ZU=; b=h7iIhVeapAB5joj9oWtqGzUvl8q4/QVcA3ZpDgLSb01kQrL4hsenaZmZaO6BQeFOwDTnm5 TLNW5dr9ZbUUEdrHxzVAJo9rCELY53fXeeHQwEAmgc+EjZT2iH+ZFUzgKHwoc6aN6sB+yO D3tji8xliZ1vNJvaAqBEvB4JmZPc8uGyE+YXIslGFwQoeSfyctAaFQDvnyWLs2JX36g9VE 7lJvM0cEmusatW4mZ6zzfuub7LSjJxOf3721UIXZkfU0jJ12eJEu8hll+wnFbmD7/dpblx b5Wv0JgooQWYftgqWq85sxIorLDcBONE4ajW5VJieYMbrMclrJ9QEhn7sTa5mg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1685475007; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=2uQsYCyOprB9lncmssuCpU619HEKxhlNXL0lD5H84ZU=; b=NqS92V2IG2xp+YQnyZs92yHom27omIAGXXduOnzvM9kmEatp7zqo/EQ0dUEVVK4lgAaV+7 9qnaCn8zYYKLQPRKLYQ0bY2iI+itF7rcLSqVMClK8F90cMqMOJm76dPJkm61i+eGtmoJDr Y6R38SLPpUjsCdbeBUXl3eoyMSmtCOwggRcV+OfwvIQRovslxG5o8GW+iFkO8RjfgRBbS1 btJiib1KtilA8Dn08yCN5mFMEtD9BGXHzuZVjyY6EltvDbzH6r4lNiUrn6YS3SUY3W8+gp m+hbIKrPoTi9DC1YNU7mX6XyhyAA5LUVFLMJiIyZ5m/duVOcDLEOKuOSUYyzZA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1685475007; a=rsa-sha256; cv=none; b=LGjLWxC4MnzVBTV37CaoPq26wo9+kVKfANr+ZybU6ZViqzqSWLvuiw01ds0uGHj99C8gy/ 0x3C2H1+fHPe75zCd67XaDmwMLeShfFPE49otBlfzXGU7JVPwd6zFbN6Qw3h20AV45GRMT Ycl3wYgNk3sT3IV/Sv9s/Epcgo4rP7n6yNwXGRY0luNuFDRD81fzuI5lsdk9Rku+N/7hnU K/8JY0XJ+iTqtYyxy+/SBB062ara1etjY8XX8CiNs2swDtjotYxdotbvTRZHIYywqDXSSK gG+YjKi/Vdv9NiafUZs3tYUn5vXLLUX18kdt7h/DafOkx1b7J9XiixHRRQZoFA== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4QW2Xv551jz15d6; Tue, 30 May 2023 19:30:07 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 34UJU7CV048054; Tue, 30 May 2023 19:30:07 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 34UJU7ce048051; Tue, 30 May 2023 19:30:07 GMT (envelope-from git) Date: Tue, 30 May 2023 19:30:07 GMT Message-Id: <202305301930.34UJU7ce048051@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Mark Johnston Subject: git: 4e78addbeff9 - main - buf: Make the number of pbufs slightly more dynamic List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-main@freebsd.org X-BeenThere: dev-commits-src-main@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: markj X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 4e78addbeff902aabaa87fdaafbd962f90720d69 Auto-Submitted: auto-generated X-ThisMailContainsUnwantedMimeParts: N The branch main has been updated by markj: URL: https://cgit.FreeBSD.org/src/commit/?id=4e78addbeff902aabaa87fdaafbd962f90720d69 commit 4e78addbeff902aabaa87fdaafbd962f90720d69 Author: Mark Johnston AuthorDate: 2023-05-30 19:11:32 +0000 Commit: Mark Johnston CommitDate: 2023-05-30 19:11:32 +0000 buf: Make the number of pbufs slightly more dynamic Various subsystems pre-allocate a set of pbufs, allocated to implement I/O operations. pbuf allocations are transient, unlike most buf allocations. Most subsystems preallocate nswbuf or nswbuf/2 pbufs each. The preallocation ensures that pbuf allocation will succeed in low memory conditions, which might help avoid deadlocks. Currently we initialize nswbuf = min(nbuf / 4, 256). nbuf/4 > 256 on anything but the smallest systems. For example, nswbuf is 256 in a VM with 128MB of memory. In this configuration, a firecracker VM with one CPU preallocates over 900 pbufs. This consumes 2MB of RAM and adds several milliseconds to the kernel's (very small) boot time. Scale nswbuf by ncpu in the common case. I think this makes more sense than scaling by the amount of RAM, since pbuf allocations are transient and aren't used for caching. With the change, we get nswbuf=256 with 8 CPUs. With fewer than 8 CPUs we'll preallocate fewer pbufs than before, and with more we'll preallocate more. Event: BSDCan 2023 Reported by: cperciva Reviewed by: glebius, kib MFC after: 2 months Differential Revision: https://reviews.freebsd.org/D40216 --- sys/kern/vfs_bio.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/sys/kern/vfs_bio.c b/sys/kern/vfs_bio.c index 177b201e4819..a76ea26d8859 100644 --- a/sys/kern/vfs_bio.c +++ b/sys/kern/vfs_bio.c @@ -1168,7 +1168,12 @@ kern_vfs_bio_buffer_alloc(caddr_t v, long physmem_est) } if (nswbuf == 0) { - nswbuf = min(nbuf / 4, 256); + /* + * Pager buffers are allocated for short periods, so scale the + * number of reserved buffers based on the number of CPUs rather + * than amount of memory. + */ + nswbuf = min(nbuf / 4, 32 * mp_ncpus); if (nswbuf < NSWBUF_MIN) nswbuf = NSWBUF_MIN; }