From nobody Fri Jan 20 03:23:33 2023 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4NylFd6lPVz30vfc; Fri, 20 Jan 2023 03:23:33 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4NylFd5sbJz3nTR; Fri, 20 Jan 2023 03:23:33 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1674185013; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=G3IWk3H61+p5Tgu3V0P1rolnjkS7NrmQkUhV9WeJASw=; b=aYR2xgg+PW/ue4KKHLI9N4rjQeO5YGL4kjXnF0bIjK8qlogI+i9LJRTu9dYQfZiRJEbFl9 9XXvYG76X0+F4TbOOMYy/kWMr78GYqtNzbUQNdK+IHnfLA2LQmgtK3wqFQwyO/20uuic4w V7/zkOEaf7bv3GgtpLal+ST4vZS3DB3Btvli29gywvDtpah8jGlAIHAsmCeJSEw/DAsu4k OXLKU+V9++DcPUs63lqMq920KhoPu3E7AXX/oV4iJjP4hyo1VZ3NWRIy30HD64Q5GOALvt +s6zzkyVqeuuef0rcD8MrVvjPJRkD33+TnZVNmQ/RtnAizGOdRn3iuBTC0z5lQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1674185013; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=G3IWk3H61+p5Tgu3V0P1rolnjkS7NrmQkUhV9WeJASw=; b=bCfIZ4Q1CIt78qrM2sC6gvl6xLkhOqOoXsvoEB0ffZviZJAywBdp3ol+9JGVYgR7dvK2lc huRjp0jMwn1iBZ0+KeEiDMn2TJET+YJ8A75/g3NDv744N/kBiSq1YM67e7U8cg3HCDRsE5 GtpBSPNw0QNAyO4Mzr0m7HfCPJwbl0BvjeRWC81scrRMJpZXvOu+R4SnXyR+Mkzg7hbYmg bltVIMOdum8weaV1UbsWTfYOr/r2AdwcN/6QEGO3QUy9PbPQeA2MG/t+fVU69wU4N0wCpm wwqqFtJOUwJIuZ4WAnCpw1Tn6N00pRzAyUdTEcOmBmXdCS1sJ9Eq37PcFrshbg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1674185013; a=rsa-sha256; cv=none; b=oYE2h/Mk0UskuSlxIGe8V6acb3MWxkXryvKb8mnubQgmORvXmmMPwOXnNbMREmqjdhBA77 975Wkb6ecsrkIe3anTlmyk5Z3E1g0dHC+dYINgaoHwAePKAaTwBdOtf+LPfqzjjy+kSPin ESHiKY8fIlPmlbVrGdENBaS6Aj1rcJfG19ASPo7bz4Mo0KrQOlihQBQSwWRcJHQbzT2ewU qSRz+XwmhypKrKPtqv2y1S5Zb7A7acnKyOnkDLWwWdKGHWv+B2fSmjd96NjyWxAyplFvoS IxQRBWsvQEuAyaJ6zUH5jFGWoUAKTMDqL2K/FC1B9T91ceo8jHtddoQWN60hxg== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4NylFd4yDDzFyq; Fri, 20 Jan 2023 03:23:33 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 30K3NXCM042319; Fri, 20 Jan 2023 03:23:33 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 30K3NXN1042318; Fri, 20 Jan 2023 03:23:33 GMT (envelope-from git) Date: Fri, 20 Jan 2023 03:23:33 GMT Message-Id: <202301200323.30K3NXN1042318@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Konstantin Belousov Subject: git: 06e03b4b0d24 - stable/13 - tmpfs: for used pages, account really allocated pages, instead of file sizes List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-all@freebsd.org X-BeenThere: dev-commits-src-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: kib X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: 06e03b4b0d24bc0c513eb7dc5651664565d0c219 Auto-Submitted: auto-generated X-ThisMailContainsUnwantedMimeParts: N The branch stable/13 has been updated by kib: URL: https://cgit.FreeBSD.org/src/commit/?id=06e03b4b0d24bc0c513eb7dc5651664565d0c219 commit 06e03b4b0d24bc0c513eb7dc5651664565d0c219 Author: Konstantin Belousov AuthorDate: 2022-10-20 13:17:43 +0000 Commit: Konstantin Belousov CommitDate: 2023-01-20 03:19:50 +0000 tmpfs: for used pages, account really allocated pages, instead of file sizes PR: 223015 Tested by: pho (cherry picked from commit 37aea2649ff707f23d35309d882b38e9ac818e42) --- sys/fs/tmpfs/tmpfs.h | 18 ++++++- sys/fs/tmpfs/tmpfs_subr.c | 118 ++++++++++++++++++++++++++++++++++++++++---- sys/fs/tmpfs/tmpfs_vfsops.c | 6 ++- sys/fs/tmpfs/tmpfs_vnops.c | 17 +++++-- sys/kern/uipc_shm.c | 2 +- 5 files changed, 144 insertions(+), 17 deletions(-) diff --git a/sys/fs/tmpfs/tmpfs.h b/sys/fs/tmpfs/tmpfs.h index 40f32ed62a7b..30c80ad70d48 100644 --- a/sys/fs/tmpfs/tmpfs.h +++ b/sys/fs/tmpfs/tmpfs.h @@ -148,6 +148,7 @@ RB_HEAD(tmpfs_dir, tmpfs_dirent); * (i) tn_interlock * (m) tmpfs_mount tm_allnode_lock * (c) stable after creation + * (v) tn_reg.tn_aobj vm_object lock */ struct tmpfs_node { /* @@ -299,6 +300,7 @@ struct tmpfs_node { */ vm_object_t tn_aobj; /* (c) */ struct tmpfs_mount *tn_tmp; /* (c) */ + vm_pindex_t tn_pages; /* (v) */ } tn_reg; } tn_spec; /* (v) */ }; @@ -532,7 +534,8 @@ VM_TO_TMPFS_VP(vm_object_t obj) { struct tmpfs_node *node; - MPASS((obj->flags & OBJ_TMPFS) != 0); + if ((obj->flags & OBJ_TMPFS) == 0) + return (NULL); /* * swp_priv is the back-pointer to the tmpfs node, if any, @@ -544,6 +547,19 @@ VM_TO_TMPFS_VP(vm_object_t obj) return (node->tn_vnode); } +static inline struct tmpfs_mount * +VM_TO_TMPFS_MP(vm_object_t obj) +{ + struct tmpfs_node *node; + + if ((obj->flags & OBJ_TMPFS) == 0) + return (NULL); + + node = obj->un_pager.swp.swp_priv; + MPASS(node->tn_type == VREG); + return (node->tn_reg.tn_tmp); +} + static inline struct tmpfs_mount * VFS_TO_TMPFS(struct mount *mp) { diff --git a/sys/fs/tmpfs/tmpfs_subr.c b/sys/fs/tmpfs/tmpfs_subr.c index 7125b16dea33..1c0b4406e460 100644 --- a/sys/fs/tmpfs/tmpfs_subr.c +++ b/sys/fs/tmpfs/tmpfs_subr.c @@ -214,6 +214,87 @@ tmpfs_pager_getvp(vm_object_t object, struct vnode **vpp, bool *vp_heldp) VM_OBJECT_RUNLOCK(object); } +static void +tmpfs_pager_freespace(vm_object_t obj, vm_pindex_t start, vm_size_t size) +{ + struct tmpfs_node *node; + struct tmpfs_mount *tm; + vm_size_t c; + + swap_pager_freespace(obj, start, size, &c); + if ((obj->flags & OBJ_TMPFS) == 0 || c == 0) + return; + + node = obj->un_pager.swp.swp_priv; + MPASS(node->tn_type == VREG); + tm = node->tn_reg.tn_tmp; + + KASSERT(tm->tm_pages_used >= c, + ("tmpfs tm %p pages %jd free %jd", tm, + (uintmax_t)tm->tm_pages_used, (uintmax_t)c)); + atomic_add_long(&tm->tm_pages_used, -c); + KASSERT(node->tn_reg.tn_pages >= c, + ("tmpfs node %p pages %jd free %jd", node, + (uintmax_t)node->tn_reg.tn_pages, (uintmax_t)c)); + node->tn_reg.tn_pages -= c; +} + +static void +tmpfs_page_inserted(vm_object_t obj, vm_page_t m) +{ + struct tmpfs_node *node; + struct tmpfs_mount *tm; + + if ((obj->flags & OBJ_TMPFS) == 0) + return; + + node = obj->un_pager.swp.swp_priv; + MPASS(node->tn_type == VREG); + tm = node->tn_reg.tn_tmp; + + if (!vm_pager_has_page(obj, m->pindex, NULL, NULL)) { + atomic_add_long(&tm->tm_pages_used, 1); + node->tn_reg.tn_pages += 1; + } +} + +static void +tmpfs_page_removed(vm_object_t obj, vm_page_t m) +{ + struct tmpfs_node *node; + struct tmpfs_mount *tm; + + if ((obj->flags & OBJ_TMPFS) == 0) + return; + + node = obj->un_pager.swp.swp_priv; + MPASS(node->tn_type == VREG); + tm = node->tn_reg.tn_tmp; + + if (!vm_pager_has_page(obj, m->pindex, NULL, NULL)) { + KASSERT(tm->tm_pages_used >= 1, + ("tmpfs tm %p pages %jd free 1", tm, + (uintmax_t)tm->tm_pages_used)); + atomic_add_long(&tm->tm_pages_used, -1); + KASSERT(node->tn_reg.tn_pages >= 1, + ("tmpfs node %p pages %jd free 1", node, + (uintmax_t)node->tn_reg.tn_pages)); + node->tn_reg.tn_pages -= 1; + } +} + +static boolean_t +tmpfs_can_alloc_page(vm_object_t obj, vm_pindex_t pindex) +{ + struct tmpfs_mount *tm; + + tm = VM_TO_TMPFS_MP(obj); + if (tm == NULL || vm_pager_has_page(obj, pindex, NULL, NULL) || + tm->tm_pages_max == 0) + return (true); + return (tm->tm_pages_max > atomic_load_long(&tm->tm_pages_used)); +} + struct pagerops tmpfs_pager_ops = { .pgo_kvme_type = KVME_TYPE_VNODE, .pgo_alloc = tmpfs_pager_alloc, @@ -222,6 +303,10 @@ struct pagerops tmpfs_pager_ops = { .pgo_release_writecount = tmpfs_pager_release_writecount, .pgo_mightbedirty = vm_object_mightbedirty_, .pgo_getvp = tmpfs_pager_getvp, + .pgo_freespace = tmpfs_pager_freespace, + .pgo_page_inserted = tmpfs_page_inserted, + .pgo_page_removed = tmpfs_page_removed, + .pgo_can_alloc_page = tmpfs_can_alloc_page, }; static int @@ -576,6 +661,7 @@ tmpfs_alloc_node(struct mount *mp, struct tmpfs_mount *tmp, enum vtype type, nnode->tn_reg.tn_aobj->un_pager.swp.swp_priv = nnode; vm_object_set_flag(nnode->tn_reg.tn_aobj, OBJ_TMPFS); nnode->tn_reg.tn_tmp = tmp; + nnode->tn_reg.tn_pages = 0; break; default: @@ -665,11 +751,31 @@ tmpfs_free_node_locked(struct tmpfs_mount *tmp, struct tmpfs_node *node, switch (node->tn_type) { case VREG: uobj = node->tn_reg.tn_aobj; - if (uobj != NULL && uobj->size != 0) - atomic_subtract_long(&tmp->tm_pages_used, uobj->size); + node->tn_reg.tn_aobj = NULL; + if (uobj != NULL) { + VM_OBJECT_WLOCK(uobj); + KASSERT((uobj->flags & OBJ_TMPFS) != 0, + ("tmpfs node %p uobj %p not tmpfs", node, uobj)); + vm_object_clear_flag(uobj, OBJ_TMPFS); + KASSERT(tmp->tm_pages_used >= node->tn_reg.tn_pages, + ("tmpfs tmp %p node %p pages %jd free %jd", tmp, + node, (uintmax_t)tmp->tm_pages_used, + (uintmax_t)node->tn_reg.tn_pages)); + atomic_add_long(&tmp->tm_pages_used, + -node->tn_reg.tn_pages); + VM_OBJECT_WUNLOCK(uobj); + } tmpfs_free_tmp(tmp); + + /* + * vm_object_deallocate() must not be called while + * owning tm_allnode_lock, because deallocate might + * sleep. Call it after tmpfs_free_tmp() does the + * unlock. + */ if (uobj != NULL) vm_object_deallocate(uobj); + break; case VLNK: tmpfs_free_tmp(tmp); @@ -1705,7 +1811,6 @@ tmpfs_dir_whiteout_remove(struct vnode *dvp, struct componentname *cnp) int tmpfs_reg_resize(struct vnode *vp, off_t newsize, boolean_t ignerr) { - struct tmpfs_mount *tmp; struct tmpfs_node *node; vm_object_t uobj; vm_pindex_t idx, newpages, oldpages; @@ -1717,7 +1822,6 @@ tmpfs_reg_resize(struct vnode *vp, off_t newsize, boolean_t ignerr) node = VP_TO_TMPFS_NODE(vp); uobj = node->tn_reg.tn_aobj; - tmp = VFS_TO_TMPFS(vp->v_mount); /* * Convert the old and new sizes to the number of pages needed to @@ -1735,10 +1839,6 @@ tmpfs_reg_resize(struct vnode *vp, off_t newsize, boolean_t ignerr) return (0); } - if (newpages > oldpages && - !tmpfs_pages_check_avail(tmp, newpages - oldpages)) - return (ENOSPC); - VM_OBJECT_WLOCK(uobj); if (newsize < oldsize) { /* @@ -1764,8 +1864,6 @@ tmpfs_reg_resize(struct vnode *vp, off_t newsize, boolean_t ignerr) uobj->size = newpages; VM_OBJECT_WUNLOCK(uobj); - atomic_add_long(&tmp->tm_pages_used, newpages - oldpages); - node->tn_size = newsize; return (0); } diff --git a/sys/fs/tmpfs/tmpfs_vfsops.c b/sys/fs/tmpfs/tmpfs_vfsops.c index 7cba983a1847..19d67b79c2e3 100644 --- a/sys/fs/tmpfs/tmpfs_vfsops.c +++ b/sys/fs/tmpfs/tmpfs_vfsops.c @@ -557,7 +557,11 @@ tmpfs_free_tmp(struct tmpfs_mount *tmp) TMPFS_UNLOCK(tmp); mtx_destroy(&tmp->tm_allnode_lock); - MPASS(tmp->tm_pages_used == 0); + /* + * We cannot assert that tmp->tm_pages_used == 0 there, + * because tmpfs vm_objects might be still mapped by some + * process and outlive the mount due to reference counting. + */ MPASS(tmp->tm_nodes_inuse == 0); free(tmp, M_TMPFSMNT); diff --git a/sys/fs/tmpfs/tmpfs_vnops.c b/sys/fs/tmpfs/tmpfs_vnops.c index 788e966fedb7..661004595f18 100644 --- a/sys/fs/tmpfs/tmpfs_vnops.c +++ b/sys/fs/tmpfs/tmpfs_vnops.c @@ -444,7 +444,6 @@ tmpfs_stat(struct vop_stat_args *v) { struct vnode *vp = v->a_vp; struct stat *sb = v->a_sb; - vm_object_t obj; struct tmpfs_node *node; int error; @@ -477,8 +476,16 @@ tmpfs_stat(struct vop_stat_args *v) sb->st_flags = node->tn_flags; sb->st_gen = node->tn_gen; if (vp->v_type == VREG) { - obj = node->tn_reg.tn_aobj; - sb->st_blocks = (u_quad_t)obj->resident_page_count * PAGE_SIZE; +#ifdef __ILP32__ + vm_object_t obj = node->tn_reg.tn_aobj; + + /* Handle torn read */ + VM_OBJECT_RLOCK(obj); +#endif + sb->st_blocks = ptoa(node->tn_reg.tn_pages); +#ifdef __ILP32__ + VM_OBJECT_RUNLOCK(obj); +#endif } else { sb->st_blocks = node->tn_size; } @@ -517,7 +524,9 @@ tmpfs_getattr(struct vop_getattr_args *v) node->tn_rdev : NODEV; if (vp->v_type == VREG) { obj = node->tn_reg.tn_aobj; - vap->va_bytes = (u_quad_t)obj->resident_page_count * PAGE_SIZE; + VM_OBJECT_RLOCK(obj); + vap->va_bytes = ptoa(node->tn_reg.tn_pages); + VM_OBJECT_RUNLOCK(obj); } else { vap->va_bytes = node->tn_size; } diff --git a/sys/kern/uipc_shm.c b/sys/kern/uipc_shm.c index 4f06d9bea40e..a1f26b40f343 100644 --- a/sys/kern/uipc_shm.c +++ b/sys/kern/uipc_shm.c @@ -225,7 +225,7 @@ uiomove_object_page(vm_object_t obj, size_t len, struct uio *uio) printf("uiomove_object: vm_obj %p idx %jd " "pager error %d\n", obj, idx, rv); } - return (EIO); + return (rv == VM_PAGER_AGAIN ? ENOSPC : EIO); } VM_OBJECT_WUNLOCK(obj);