git: f5973f2e90 - main - Add EN-24:09.
- Go to: [ bottom of page ] [ top of archives ] [ this month ]
Date: Wed, 24 Apr 2024 20:26:13 UTC
The branch main has been updated by gordon: URL: https://cgit.FreeBSD.org/doc/commit/?id=f5973f2e90e430ed97d65de8dd5230f6c4405f0d commit f5973f2e90e430ed97d65de8dd5230f6c4405f0d Author: Gordon Tetlow <gordon@FreeBSD.org> AuthorDate: 2024-04-24 20:25:48 +0000 Commit: Gordon Tetlow <gordon@FreeBSD.org> CommitDate: 2024-04-24 20:25:48 +0000 Add EN-24:09. Approved by: so --- website/data/security/errata.toml | 4 + .../security/advisories/FreeBSD-EN-24:09.zfs.asc | 144 ++++++++++ website/static/security/patches/EN-24:09/zfs.patch | 316 +++++++++++++++++++++ .../static/security/patches/EN-24:09/zfs.patch.asc | 16 ++ 4 files changed, 480 insertions(+) diff --git a/website/data/security/errata.toml b/website/data/security/errata.toml index 50d0a89e3a..702576a943 100644 --- a/website/data/security/errata.toml +++ b/website/data/security/errata.toml @@ -1,6 +1,10 @@ # Sort errata notices by year, month and day # $FreeBSD$ +[[notices]] +name = "FreeBSD-EN-24:09.zfs" +date = "2024-04-24" + [[notices]] name = "FreeBSD-EN-24:08.kerberos" date = "2024-03-28" diff --git a/website/static/security/advisories/FreeBSD-EN-24:09.zfs.asc b/website/static/security/advisories/FreeBSD-EN-24:09.zfs.asc new file mode 100644 index 0000000000..3a3b203d3a --- /dev/null +++ b/website/static/security/advisories/FreeBSD-EN-24:09.zfs.asc @@ -0,0 +1,144 @@ +-----BEGIN PGP SIGNED MESSAGE----- +Hash: SHA512 + +============================================================================= +FreeBSD-EN-24:09.zfs Errata Notice + The FreeBSD Project + +Topic: High CPU usage by kernel threads related to ZFS + +Category: contrib +Module: zfs +Announced: 2024-04-24 +Affects: FreeBSD 13.3 +Corrected: 2024-04-12 13:00:11 UTC (stable/13, 13-STABLE) + 2024-04-24 20:21:10 UTC (releng/13.3, 13.3-RELEASE-p2) + +For general information regarding FreeBSD Errata Notices and Security +Advisories, including descriptions of the fields above, security +branches, and the following sections, please visit +<URL:https://security.FreeBSD.org/>. + +I. Background + +ZFS is an advanced and scalable file system originally developed by Sun +Microsystems for its Solaris operating system. ZFS was integrated as part of +the FreeBSD starting with FreeBSD 7.0, and it has since become a prominent +and preferred choice for storage management. + +II. Problem Description + +Because ZFS may consume large amounts of RAM to cache various types of +filesystem objects, it continuously monitors system RAM available to decide +whether to shrink its caches. Some caches are shrunk using a dedicated +thread, to which work is dispatched asynchronously. + +In some cases, the cache shrinking logic may dispatch excessive amounts of +work to the "ARC pruning" thread, causing it to continue attempting to shrink +caches even after resource shortages are resolved. + +III. Impact + +The bug manifests as a kernel thread, "arc_prune", consuming 100% of a CPU core +for indefinite periods, even while the system is otherwise idle. This behavior +also impacts workloads running on the system, by reducing available CPU +resources and by triggering lock contention in the kernel, in particular with +the "vnlru" process whose function is to recycle vnodes (structures representing +files, whether opened or cached), a mechanism frequently triggered by intensive +filesystem workloads. + +IV. Workaround + +No workaround is available. Systems not using ZFS are unaffected. + +V. Solution + +Upgrade your system to a supported FreeBSD stable or release / security branch +(releng) dated after the correction date. A reboot is required following the +upgrade. + +Perform one of the following: + +1) To update your system via a binary patch: + +Systems running a RELEASE version of FreeBSD on the amd64 or arm64 platforms, +or the i386 platform on FreeBSD 13 and earlier, can be updated via +the freebsd-update(8) utility: + +# freebsd-update fetch +# freebsd-update install +# reboot + +2) To update your system via a source code patch: + +The following patches have been verified to apply to the applicable +FreeBSD release branches. + +a) Download the relevant patch from the location below, and verify the +detached PGP signature using your PGP utility. + +# fetch https://security.FreeBSD.org/patches/EN-24:09/zfs.patch +# fetch https://security.FreeBSD.org/patches/EN-24:09/zfs.patch.asc +# gpg --verify zfs.patch.asc + +b) Apply the patch. Execute the following commands as root: + +# cd /usr/src +# patch < /path/to/patch + +c) Recompile your kernel as described in +<URL:https://www.FreeBSD.org/handbook/kernelconfig.html> and reboot the +system. + +VI. Correction details + +This issue is corrected as of the corresponding Git commit hash or Subversion +revision number in the following stable and release branches: + +Branch/path Hash Revision +- ------------------------------------------------------------------------- +stable/13/ 330954bdb822 stable/13-n257698 +releng/13.3/ 266b3bd3f26d releng/13.3-n257432 +- ------------------------------------------------------------------------- + +Run the following command to see which files were modified by a +particular commit: + +# git show --stat <commit hash> + +Or visit the following URL, replacing NNNNNN with the hash: + +<URL:https://cgit.freebsd.org/src/commit/?id=NNNNNN> + +To determine the commit count in a working tree (for comparison against +nNNNNNN in the table above), run: + +# git rev-list --count --first-parent HEAD + +VII. References + +See problem reports +<URL:https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=274698> and +<URL:https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=275594>. + +See also the previous, similar errata notice issued for FreeBSD 14.0: +<URL:https://security.FreeBSD.org/advisories/FreeBSD-EN-23:18.openzfs.asc>. + +The latest revision of this advisory is available at +<URL:https://security.FreeBSD.org/advisories/FreeBSD-EN-24:09.zfs.asc> +-----BEGIN PGP SIGNATURE----- + +iQIzBAEBCgAdFiEEthUnfoEIffdcgYM7bljekB8AGu8FAmYpapQACgkQbljekB8A +Gu8gBxAAiuUNqeGaKNQ1XbV0kSucwnae5uOrQmthHQBY98PJJKUZpm1RTt/FnBB7 +qPxEY5vFRcGgZ43GVlnmfmH/EmqOg6WPpsgKfdq1XTy/ERU815JOsD+wKUWa/9Ia +g67pnl8HPMSF5eZ1FreWfzNsWmxakiDLg2VXtFx7x3+qocifD/WwGvDTjdDBzzyK ++cIrBqvTlbOCRdHzl49wmNLz46ha5bmxTb7MzXB3jIQ1v+PZ71biyQxBZTrZgR6S +La8oVe4Kj2lJTJw5S2xvsoyo5PzqmPCyD1m22fzgKTyaAUCXiioUUQDuFTxu9rhW +I3lSvqdIRw28yRFjGslxlq9x1vShQTw3ILcH31ucxKUNow7hlDz4Ow2NzqXhSjxN +RMGamxLTA5BcNCR4/DexAjfeh6OKnCG7n0ntlhxI0LWGr4ceT3/ySck7xhCNCSm1 +Ze/Gf9/j4+zR2jyauRANkITPkVHUV79/Sgjn1IlcMDLpzegH+QfQsX6CosG5uSWS +UlpK2hhCv2g3lE7XuBItz7E/8i5Nx9RZgnh047Nj3ZB/6dCauAeUYKnY5X3xJa5X +OKJWIGyJAyrCoFIg+LdBS47ggg8wswyyb1XBF2rZgZNqVmzZrJd7lBV/sjDaEC1H +13lHhIIwtpTagDAT1Nbji++IT+2DatjhLZnMQwvALno0tIE19mg= +=IgLQ +-----END PGP SIGNATURE----- diff --git a/website/static/security/patches/EN-24:09/zfs.patch b/website/static/security/patches/EN-24:09/zfs.patch new file mode 100644 index 0000000000..48e3e06a44 --- /dev/null +++ b/website/static/security/patches/EN-24:09/zfs.patch @@ -0,0 +1,316 @@ +--- sys/contrib/openzfs/include/os/linux/zfs/sys/zpl.h.orig ++++ sys/contrib/openzfs/include/os/linux/zfs/sys/zpl.h +@@ -52,7 +52,7 @@ + extern const struct file_operations zpl_dir_file_operations; + + /* zpl_super.c */ +-extern void zpl_prune_sb(int64_t nr_to_scan, void *arg); ++extern void zpl_prune_sb(uint64_t nr_to_scan, void *arg); + + extern const struct super_operations zpl_super_operations; + extern const struct export_operations zpl_export_operations; +--- sys/contrib/openzfs/include/sys/arc.h.orig ++++ sys/contrib/openzfs/include/sys/arc.h +@@ -81,7 +81,7 @@ + typedef void arc_read_done_func_t(zio_t *zio, const zbookmark_phys_t *zb, + const blkptr_t *bp, arc_buf_t *buf, void *priv); + typedef void arc_write_done_func_t(zio_t *zio, arc_buf_t *buf, void *priv); +-typedef void arc_prune_func_t(int64_t bytes, void *priv); ++typedef void arc_prune_func_t(uint64_t bytes, void *priv); + + /* Shared module parameters */ + extern int zfs_arc_average_blocksize; +--- sys/contrib/openzfs/include/sys/arc_impl.h.orig ++++ sys/contrib/openzfs/include/sys/arc_impl.h +@@ -994,7 +994,6 @@ + + extern void arc_lowmem_init(void); + extern void arc_lowmem_fini(void); +-extern void arc_prune_async(int64_t); + extern int arc_memory_throttle(spa_t *spa, uint64_t reserve, uint64_t txg); + extern uint64_t arc_free_memory(void); + extern int64_t arc_available_memory(void); +--- sys/contrib/openzfs/module/os/freebsd/zfs/arc_os.c.orig ++++ sys/contrib/openzfs/module/os/freebsd/zfs/arc_os.c +@@ -51,11 +51,6 @@ + #include <sys/vm.h> + #include <sys/vmmeter.h> + +-#if __FreeBSD_version >= 1300139 +-static struct sx arc_vnlru_lock; +-static struct vnode *arc_vnlru_marker; +-#endif +- + extern struct vfsops zfs_vfsops; + + uint_t zfs_arc_free_target = 0; +@@ -151,53 +146,6 @@ + return (MAX(allmem * 5 / 8, size)); + } + +-/* +- * Helper function for arc_prune_async() it is responsible for safely +- * handling the execution of a registered arc_prune_func_t. +- */ +-static void +-arc_prune_task(void *arg) +-{ +- int64_t nr_scan = (intptr_t)arg; +- +-#ifndef __ILP32__ +- if (nr_scan > INT_MAX) +- nr_scan = INT_MAX; +-#endif +- +-#if __FreeBSD_version >= 1300139 +- sx_xlock(&arc_vnlru_lock); +- vnlru_free_vfsops(nr_scan, &zfs_vfsops, arc_vnlru_marker); +- sx_xunlock(&arc_vnlru_lock); +-#else +- vnlru_free(nr_scan, &zfs_vfsops); +-#endif +-} +- +-/* +- * Notify registered consumers they must drop holds on a portion of the ARC +- * buffered they reference. This provides a mechanism to ensure the ARC can +- * honor the arc_meta_limit and reclaim otherwise pinned ARC buffers. This +- * is analogous to dnlc_reduce_cache() but more generic. +- * +- * This operation is performed asynchronously so it may be safely called +- * in the context of the arc_reclaim_thread(). A reference is taken here +- * for each registered arc_prune_t and the arc_prune_task() is responsible +- * for releasing it once the registered arc_prune_func_t has completed. +- */ +-void +-arc_prune_async(int64_t adjust) +-{ +- +-#ifndef __LP64__ +- if (adjust > INTPTR_MAX) +- adjust = INTPTR_MAX; +-#endif +- taskq_dispatch(arc_prune_taskq, arc_prune_task, +- (void *)(intptr_t)adjust, TQ_SLEEP); +- ARCSTAT_BUMP(arcstat_prune); +-} +- + uint64_t + arc_all_memory(void) + { +@@ -248,10 +196,6 @@ + { + arc_event_lowmem = EVENTHANDLER_REGISTER(vm_lowmem, arc_lowmem, NULL, + EVENTHANDLER_PRI_FIRST); +-#if __FreeBSD_version >= 1300139 +- arc_vnlru_marker = vnlru_alloc_marker(); +- sx_init(&arc_vnlru_lock, "arc vnlru lock"); +-#endif + } + + void +@@ -259,12 +203,6 @@ + { + if (arc_event_lowmem != NULL) + EVENTHANDLER_DEREGISTER(vm_lowmem, arc_event_lowmem); +-#if __FreeBSD_version >= 1300139 +- if (arc_vnlru_marker != NULL) { +- vnlru_free_marker(arc_vnlru_marker); +- sx_destroy(&arc_vnlru_lock); +- } +-#endif + } + + void +--- sys/contrib/openzfs/module/os/freebsd/zfs/zfs_vfsops.c.orig ++++ sys/contrib/openzfs/module/os/freebsd/zfs/zfs_vfsops.c +@@ -2097,6 +2097,26 @@ + #endif + } + ++#if __FreeBSD_version >= 1300139 ++static struct sx zfs_vnlru_lock; ++static struct vnode *zfs_vnlru_marker; ++#endif ++static arc_prune_t *zfs_prune; ++ ++static void ++zfs_prune_task(uint64_t nr_to_scan, void *arg __unused) ++{ ++ if (nr_to_scan > INT_MAX) ++ nr_to_scan = INT_MAX; ++#if __FreeBSD_version >= 1300139 ++ sx_xlock(&zfs_vnlru_lock); ++ vnlru_free_vfsops(nr_to_scan, &zfs_vfsops, zfs_vnlru_marker); ++ sx_xunlock(&zfs_vnlru_lock); ++#else ++ vnlru_free(nr_to_scan, &zfs_vfsops); ++#endif ++} ++ + void + zfs_init(void) + { +@@ -2123,11 +2143,23 @@ + dmu_objset_register_type(DMU_OST_ZFS, zpl_get_file_info); + + zfsvfs_taskq = taskq_create("zfsvfs", 1, minclsyspri, 0, 0, 0); ++ ++#if __FreeBSD_version >= 1300139 ++ zfs_vnlru_marker = vnlru_alloc_marker(); ++ sx_init(&zfs_vnlru_lock, "zfs vnlru lock"); ++#endif ++ zfs_prune = arc_add_prune_callback(zfs_prune_task, NULL); + } + + void + zfs_fini(void) + { ++ arc_remove_prune_callback(zfs_prune); ++#if __FreeBSD_version >= 1300139 ++ vnlru_free_marker(zfs_vnlru_marker); ++ sx_destroy(&zfs_vnlru_lock); ++#endif ++ + taskq_destroy(zfsvfs_taskq); + zfsctl_fini(); + zfs_znode_fini(); +--- sys/contrib/openzfs/module/os/linux/zfs/arc_os.c.orig ++++ sys/contrib/openzfs/module/os/linux/zfs/arc_os.c +@@ -491,57 +491,6 @@ + } + #endif /* _KERNEL */ + +-/* +- * Helper function for arc_prune_async() it is responsible for safely +- * handling the execution of a registered arc_prune_func_t. +- */ +-static void +-arc_prune_task(void *ptr) +-{ +- arc_prune_t *ap = (arc_prune_t *)ptr; +- arc_prune_func_t *func = ap->p_pfunc; +- +- if (func != NULL) +- func(ap->p_adjust, ap->p_private); +- +- zfs_refcount_remove(&ap->p_refcnt, func); +-} +- +-/* +- * Notify registered consumers they must drop holds on a portion of the ARC +- * buffered they reference. This provides a mechanism to ensure the ARC can +- * honor the arc_meta_limit and reclaim otherwise pinned ARC buffers. This +- * is analogous to dnlc_reduce_cache() but more generic. +- * +- * This operation is performed asynchronously so it may be safely called +- * in the context of the arc_reclaim_thread(). A reference is taken here +- * for each registered arc_prune_t and the arc_prune_task() is responsible +- * for releasing it once the registered arc_prune_func_t has completed. +- */ +-void +-arc_prune_async(int64_t adjust) +-{ +- arc_prune_t *ap; +- +- mutex_enter(&arc_prune_mtx); +- for (ap = list_head(&arc_prune_list); ap != NULL; +- ap = list_next(&arc_prune_list, ap)) { +- +- if (zfs_refcount_count(&ap->p_refcnt) >= 2) +- continue; +- +- zfs_refcount_add(&ap->p_refcnt, ap->p_pfunc); +- ap->p_adjust = adjust; +- if (taskq_dispatch(arc_prune_taskq, arc_prune_task, +- ap, TQ_SLEEP) == TASKQID_INVALID) { +- zfs_refcount_remove(&ap->p_refcnt, ap->p_pfunc); +- continue; +- } +- ARCSTAT_BUMP(arcstat_prune); +- } +- mutex_exit(&arc_prune_mtx); +-} +- + /* BEGIN CSTYLED */ + ZFS_MODULE_PARAM(zfs_arc, zfs_arc_, shrinker_limit, INT, ZMOD_RW, + "Limit on number of pages that ARC shrinker can reclaim at once"); +--- sys/contrib/openzfs/module/os/linux/zfs/zpl_super.c.orig ++++ sys/contrib/openzfs/module/os/linux/zfs/zpl_super.c +@@ -334,7 +334,7 @@ + } + + void +-zpl_prune_sb(int64_t nr_to_scan, void *arg) ++zpl_prune_sb(uint64_t nr_to_scan, void *arg) + { + struct super_block *sb = (struct super_block *)arg; + int objects = 0; +--- sys/contrib/openzfs/module/zfs/arc.c.orig ++++ sys/contrib/openzfs/module/zfs/arc.c +@@ -868,6 +868,8 @@ + static void l2arc_hdr_arcstats_update(arc_buf_hdr_t *hdr, boolean_t incr, + boolean_t state_only); + ++static void arc_prune_async(uint64_t adjust); ++ + #define l2arc_hdr_arcstats_increment(hdr) \ + l2arc_hdr_arcstats_update((hdr), B_TRUE, B_FALSE) + #define l2arc_hdr_arcstats_decrement(hdr) \ +@@ -6521,6 +6523,56 @@ + kmem_free(p, sizeof (*p)); + } + ++/* ++ * Helper function for arc_prune_async() it is responsible for safely ++ * handling the execution of a registered arc_prune_func_t. ++ */ ++static void ++arc_prune_task(void *ptr) ++{ ++ arc_prune_t *ap = (arc_prune_t *)ptr; ++ arc_prune_func_t *func = ap->p_pfunc; ++ ++ if (func != NULL) ++ func(ap->p_adjust, ap->p_private); ++ ++ zfs_refcount_remove(&ap->p_refcnt, func); ++} ++ ++/* ++ * Notify registered consumers they must drop holds on a portion of the ARC ++ * buffers they reference. This provides a mechanism to ensure the ARC can ++ * honor the metadata limit and reclaim otherwise pinned ARC buffers. ++ * ++ * This operation is performed asynchronously so it may be safely called ++ * in the context of the arc_reclaim_thread(). A reference is taken here ++ * for each registered arc_prune_t and the arc_prune_task() is responsible ++ * for releasing it once the registered arc_prune_func_t has completed. ++ */ ++static void ++arc_prune_async(uint64_t adjust) ++{ ++ arc_prune_t *ap; ++ ++ mutex_enter(&arc_prune_mtx); ++ for (ap = list_head(&arc_prune_list); ap != NULL; ++ ap = list_next(&arc_prune_list, ap)) { ++ ++ if (zfs_refcount_count(&ap->p_refcnt) >= 2) ++ continue; ++ ++ zfs_refcount_add(&ap->p_refcnt, ap->p_pfunc); ++ ap->p_adjust = adjust; ++ if (taskq_dispatch(arc_prune_taskq, arc_prune_task, ++ ap, TQ_SLEEP) == TASKQID_INVALID) { ++ zfs_refcount_remove(&ap->p_refcnt, ap->p_pfunc); ++ continue; ++ } ++ ARCSTAT_BUMP(arcstat_prune); ++ } ++ mutex_exit(&arc_prune_mtx); ++} ++ + /* + * Notify the arc that a block was freed, and thus will never be used again. + */ diff --git a/website/static/security/patches/EN-24:09/zfs.patch.asc b/website/static/security/patches/EN-24:09/zfs.patch.asc new file mode 100644 index 0000000000..52cdb325ff --- /dev/null +++ b/website/static/security/patches/EN-24:09/zfs.patch.asc @@ -0,0 +1,16 @@ +-----BEGIN PGP SIGNATURE----- + +iQIzBAABCgAdFiEEthUnfoEIffdcgYM7bljekB8AGu8FAmYpapUACgkQbljekB8A +Gu+L3BAA1EBD3O+tAqKg9W8MvPihcNkHCVX5gDvY9p/xiN6nmo7JQsdNmoycZVjF +R07XpgAtuQ0mfw4fy/FvgAy4EK1q/SjQC/ON3zu78Hph610F2wabGW5p6qbotYl2 +P/msYGFPDqUgtw1SjVzXHKYRYjQnhQqbr2SIpH7ekOej65TiWuQSsRcl0YIQrjia +RfeH6faIXUyjNnDrlu2L71GY9XxiWR3FGXTfnXWbguz17kuWctCxT8UMfVGRoLa+ +yxzPX1cVgEW86lDtlvlnPbymR4rohGPrGlmLdHJdwY0H855ff0kQrreQzcfNYf7Y +xu/yxj51CF+ima7o8auDGPxGFzy3zkD5GQsLw2QoXG+Ad4EY4ZiaKYryqK7zug6Q +V8Im3PPx2CXHXXs3FmVYA+RVMHJo79zlDEwE450bRfTaj/NzRUlO62v6jqpUjayr +2pFVYwIyECm/qNddKY/4j/hMCjp11/H1co5uqFvXDWUfyVQB3iXHl5wjjyTdO8xw +DS9dRtNAUiCxgOHgz6k0U9C6gi6Xh8NNLE9QSU3CWpFWuTgrzIwAXYoCryg/c7J+ +17M6DnK0NN9z3ScehrVT4QgPPzxp5ziLhY84ZJ8qpCPsYV7ZR/rU9Yc/+mT5N3SE +QcJehAsEQUJjHL7EhkML61emj8i/avXau95AkCrcmHI5eLy1F+g= +=Q5E+ +-----END PGP SIGNATURE-----