From nobody Thu Jun 30 01:14:19 2022 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 0EAA887B7FA; Thu, 30 Jun 2022 01:14:20 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4LYL2g3zvyz4Vlc; Thu, 30 Jun 2022 01:14:19 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1656551659; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=0z8NwaO8qOmRO7WCHpnj3uOPzotAhhTtxBb1/pW3Hw0=; b=CvWq90ow66wh1ileOKWG0MaYan0YzghYw1WxMZb+nBdgSTCBV9CIklcjBHDzU0okv3LZCX jMzjlqiJkupx42YkcL0pt4bg6DANayIvldHQ9UimancDoKcZ837k9jB0dT1opF6OcaA/bc Ld7uniyZaVnYAhYmmUcKBYYAs5ShkPY9HwlKBTNT49oHKao1jUh+KBw+fynaZv4f/NnMIU IWNBgz+pbfb6C9/0v8XuLy/uGItLdD7VaA4d9nhJffhujrgRvZESJWRkdqQArPL1IbLVyM WVZ9WE44EmZvLZRzJ7hkdwGv5S8YMjj8JeG/n6wWVIE/dfRak+zkK+NdgjDZVg== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 6A25D1C8F6; Thu, 30 Jun 2022 01:14:19 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 25U1EJJ1097749; Thu, 30 Jun 2022 01:14:19 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 25U1EJbo097748; Thu, 30 Jun 2022 01:14:19 GMT (envelope-from git) Date: Thu, 30 Jun 2022 01:14:19 GMT Message-Id: <202206300114.25U1EJbo097748@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Alexander Motin Subject: git: efd76157eff3 - stable/12 - amd64: Stop using REP MOVSB for backward memmove()s. List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-all@freebsd.org X-BeenThere: dev-commits-src-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: mav X-Git-Repository: src X-Git-Refname: refs/heads/stable/12 X-Git-Reftype: branch X-Git-Commit: efd76157eff3c8f710df2ed9571d02f17729ff74 Auto-Submitted: auto-generated ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1656551659; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=0z8NwaO8qOmRO7WCHpnj3uOPzotAhhTtxBb1/pW3Hw0=; b=ZoLutCr2mGLSyiKB3CxVELyUtf+YURkueqM1kWkdehlGjIK7YrtR4JOT0Rk9Fq3/lAt9g4 y31jPvyAlRrYL5ykrHUi/4oB+fyTGemWT4LiJQXkLGHWmyKpxUGfHt4QExvLNpkv5Vg879 XsgVJCFetP5SZEMzCPOxKAc+fDwKGO+bNzGTH5ukCDCPEclprFoncATLrwOmGPbjui96eC FdKEk9jvKgbqIkFtdXtLYZQIOJQt0Ev9F8wnLni0WfFSOzEOla5OrQI4yB1KcX/8lnsOOf mjEAG2szmdJ6CoymLiKu0fXLDYqXiYVLTY5cVXYW3n8tXYZ1pJMTsj7UbCE8Bw== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1656551659; a=rsa-sha256; cv=none; b=NZD4aU5d3k058za4Y7vi6L555GL1luE1ozNTXUZYpGWCMA64olwnQNhsrdq1Ku2z8yjPLq ZVr5ay0WwqNfSfD5dFYdjmGNA+bHJM79928HNup3zrMPtVJ/yN7yH8fDousgcC7RckHC7K NRYlvTFHOdpXENlRYZiHHd425GzAT2vWNC4qpBh9/Yad4HUDzv44T9qxo0TlySQml2oKKU Ico6CxBl4SWrCc3/Sv4uwRc88VGbx9g83rp/J+CZYmQXcVV8WYgdy8VUK5St8GO07/8kCd ANJ/7qiHYFOi2CgwXXWGdDVdEyNsi8H7NnZ1h+I53nbysl4e4ARhevdIkDJoFQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N The branch stable/12 has been updated by mav: URL: https://cgit.FreeBSD.org/src/commit/?id=efd76157eff3c8f710df2ed9571d02f17729ff74 commit efd76157eff3c8f710df2ed9571d02f17729ff74 Author: Alexander Motin AuthorDate: 2022-06-16 18:51:50 +0000 Commit: Alexander Motin CommitDate: 2022-06-30 01:14:12 +0000 amd64: Stop using REP MOVSB for backward memmove()s. Enhanced REP MOVSB feature of CPUs starting from Ivy Bridge makes REP MOVSB the fastest way to copy memory in most of cases. However Intel Optimization Reference Manual says: "setting the DF to force REP MOVSB to copy bytes from high towards low addresses will expe- rience significant performance degradation". Measurements on Intel Cascade Lake and Alder Lake, same as on AMD Zen3 show that it can drop throughput to as low as 2.5-3.5GB/s, comparing to ~10-30GB/s of REP MOVSQ or hand-rolled loop, used for non-ERMS CPUs. This patch keeps ERMS use for forward ordered memory copies, but removes it for backward overlapped moves where it does not work. This is just a cosmetic sync with kernel, since libc does not use ERMS at this time. Reviewed by: mjg MFC after: 2 weeks (cherry picked from commit f22068d91bf53696ee13a69685e809d35776ec3f) --- lib/libc/amd64/string/memmove.S | 8 -------- 1 file changed, 8 deletions(-) diff --git a/lib/libc/amd64/string/memmove.S b/lib/libc/amd64/string/memmove.S index 4797140a1c16..fc4baaaa4aaa 100644 --- a/lib/libc/amd64/string/memmove.S +++ b/lib/libc/amd64/string/memmove.S @@ -268,13 +268,6 @@ __FBSDID("$FreeBSD$"); ALIGN_TEXT 2256: std -.if \erms == 1 - leaq -1(%rdi,%rcx),%rdi - leaq -1(%rsi,%rcx),%rsi - rep - movsb - cld -.else leaq -8(%rdi,%rcx),%rdi leaq -8(%rsi,%rcx),%rsi shrq $3,%rcx @@ -284,7 +277,6 @@ __FBSDID("$FreeBSD$"); movq %rdx,%rcx andb $7,%cl jne 2004b -.endif \end ret .endif