From nobody Thu Oct 28 19:02:12 2021 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 1B1E31825FFA; Thu, 28 Oct 2021 19:02:13 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4HgFKw5dGWz4s9Q; Thu, 28 Oct 2021 19:02:12 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 9F3851129; Thu, 28 Oct 2021 19:02:12 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 19SJ2CxT095371; Thu, 28 Oct 2021 19:02:12 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 19SJ2CGx095370; Thu, 28 Oct 2021 19:02:12 GMT (envelope-from git) Date: Thu, 28 Oct 2021 19:02:12 GMT Message-Id: <202110281902.19SJ2CGx095370@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Konstantin Belousov Subject: git: 1c69690319c5 - main - Unmap shared page manually before doing vm_map_remove() on exit or exec List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-main@freebsd.org X-BeenThere: dev-commits-src-main@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: kib X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 1c69690319c5bb7deae6ce1add6ea25bb40b3b91 Auto-Submitted: auto-generated X-ThisMailContainsUnwantedMimeParts: N The branch main has been updated by kib: URL: https://cgit.FreeBSD.org/src/commit/?id=1c69690319c5bb7deae6ce1add6ea25bb40b3b91 commit 1c69690319c5bb7deae6ce1add6ea25bb40b3b91 Author: Konstantin Belousov AuthorDate: 2021-10-20 20:32:59 +0000 Commit: Konstantin Belousov CommitDate: 2021-10-28 19:01:59 +0000 Unmap shared page manually before doing vm_map_remove() on exit or exec This allows the pmap_remove(min, max) call to see empty pmap and exploit empty pmap optimization. Reviewed by: markj Tested by: markj Sponsored by: The FreeBSD Foundation MFC after: 1 week Differential revision: https://reviews.freebsd.org/D32569 --- sys/kern/kern_exec.c | 25 +++++++++++++++++++++++++ sys/kern/kern_exit.c | 1 + sys/sys/sysent.h | 1 + 3 files changed, 27 insertions(+) diff --git a/sys/kern/kern_exec.c b/sys/kern/kern_exec.c index 52119036e95b..780b917ad21d 100644 --- a/sys/kern/kern_exec.c +++ b/sys/kern/kern_exec.c @@ -1104,6 +1104,30 @@ exec_onexec_old(struct thread *td) umtx_exec(td->td_proc); } +/* + * This is an optimization which removes the unmanaged shared page + * mapping. In combination with pmap_remove_pages(), which cleans all + * managed mappings in the process' vmspace pmap, no work will be left + * for pmap_remove(min, max). + */ +void +exec_free_abi_mappings(struct proc *p) +{ + struct vmspace *vmspace; + struct sysentvec *sv; + + vmspace = p->p_vmspace; + if (refcount_load(&vmspace->vm_refcnt) != 1) + return; + + sv = p->p_sysent; + if (sv->sv_shared_page_obj == NULL) + return; + + pmap_remove(vmspace_pmap(vmspace), sv->sv_shared_page_base, + sv->sv_shared_page_base + sv->sv_shared_page_len); +} + /* * Destroy old address space, and allocate a new stack. * The new stack is only sgrowsiz large because it is grown @@ -1146,6 +1170,7 @@ exec_new_vmspace(struct image_params *imgp, struct sysentvec *sv) vm_map_min(map) == sv_minuser && vm_map_max(map) == sv->sv_maxuser && cpu_exec_vmspace_reuse(p, map)) { + exec_free_abi_mappings(p); shmexit(vmspace); pmap_remove_pages(vmspace_pmap(vmspace)); vm_map_remove(map, vm_map_min(map), vm_map_max(map)); diff --git a/sys/kern/kern_exit.c b/sys/kern/kern_exit.c index 44203435fa68..14be2425511d 100644 --- a/sys/kern/kern_exit.c +++ b/sys/kern/kern_exit.c @@ -417,6 +417,7 @@ exit1(struct thread *td, int rval, int signo) mtx_unlock(&ppeers_lock); } + exec_free_abi_mappings(p); vmspace_exit(td); (void)acct_process(td); diff --git a/sys/sys/sysent.h b/sys/sys/sysent.h index ea96c87a79af..fc678d49750a 100644 --- a/sys/sys/sysent.h +++ b/sys/sys/sysent.h @@ -324,6 +324,7 @@ void exec_sysvec_init_secondary(struct sysentvec *sv, struct sysentvec *sv2); void exec_inittk(void); void exit_onexit(struct proc *p); +void exec_free_abi_mappings(struct proc *p); void exec_onexec_old(struct thread *td); #define INIT_SYSENTVEC(name, sv) \