From nobody Thu Jun 16 18:53:32 2022 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4DE7C83D37C; Thu, 16 Jun 2022 18:53:33 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4LPBCK01vXz52xW; Thu, 16 Jun 2022 18:53:33 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1655405613; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=uCGQJ2PwBPkk1+apEz3l+M6lDvVrR7OjhcBb8rX2Kkc=; b=Eoyc2Rp8QyYs8dZZ4QpSI48A0P5jMDcHMi116vqGHyIM19fcKI8X3oN9iW4EM60K7jLBNO 5/uMGogfS0y1+pArxGbiD+rbcmm/qqLjFUAigPxGO3cHdHTRy07cVrSi80/yTH/tjqoNYe T3bMbyAzAb2U08SU88W2FFSMkmSszgrUw5yBIjha7Wd5ojnUwiPjUeEzBp0qssy9RJbmbS Llg2P9E9/WnMNdBKrpnmYQqVzI3QU/m6k6Tax/qsIxQci/ldvjzHqxFOg5Z+gm+9wC0xWo jiJGUaWwvC04+NBJNEgc3tXMOpIlzAEaOSKNEa1ayerHLRIFwiQh08MiGF2YLg== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id D666C10BE3; Thu, 16 Jun 2022 18:53:32 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 25GIrWd2008878; Thu, 16 Jun 2022 18:53:32 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 25GIrWdL008877; Thu, 16 Jun 2022 18:53:32 GMT (envelope-from git) Date: Thu, 16 Jun 2022 18:53:32 GMT Message-Id: <202206161853.25GIrWdL008877@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Alexander Motin Subject: git: f22068d91bf5 - main - amd64: Stop using REP MOVSB for backward memmove()s. List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-all@freebsd.org X-BeenThere: dev-commits-src-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: mav X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: f22068d91bf53696ee13a69685e809d35776ec3f Auto-Submitted: auto-generated ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1655405613; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=uCGQJ2PwBPkk1+apEz3l+M6lDvVrR7OjhcBb8rX2Kkc=; b=rq5kuSdhbWsEwSoxurY99Q6hVY5ZnqZGYycPt6PikMDo/3LbxScLkOmlZbzyoRXg5+n9sd HKrC+9MbYL4WwpY21Uu/m6JYim/+ntsKcc1qHVzUXgrWUmn5s9vxbZPNqKn8Y+fhInw2kQ RRzBWFna8sOslij7uccUq83KnwPznI3Qo4t5NiUSpV9uOIrZhGvtSm1jOGrNnFdeamrX5E EYiHFIcDOMahhFYbhLCwkZFQLuzZ/rOKSmnybuNU5mhUaRjfWwsemdBhdn9HMs3Mfraa9t yzuu24EB9PZAA0+WwcGj5kPyKmk5d2FrunUu6KJsMHjGBi9499YUDM3HKZFfhw== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1655405613; a=rsa-sha256; cv=none; b=kpga6kuGzQ/vGmxNsVy/JxZ2XXNi5OBq7ktSH0AJkm445x18C8wqK14fjzvFq71woBTOLY G7GKcCsFJaqj6BHSpjtBhPcKKQx6mz3yD7TS9aBl1X0JxNwPPIngXqmXdqBnqHjaaqMyxM 8ZiZfFhS1AG3NuzVXKJXzqbWeME7Ler9rgidYYDg0MN/mXoXa5Q7oGgruSzJc0OcsXAd/D h4ByOO7FEkIb60Deo9fyl1/0pHTe9obPGhLSkgy22hWKqs/WNIX7MOCFdAFAlE4NUWhJDD 4mGmwbIqx8c8GF/uIcHGR+v0JW//5EwiSfmyN2PbQ50yVp2kBpJYmqOsC3uTrw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N The branch main has been updated by mav: URL: https://cgit.FreeBSD.org/src/commit/?id=f22068d91bf53696ee13a69685e809d35776ec3f commit f22068d91bf53696ee13a69685e809d35776ec3f Author: Alexander Motin AuthorDate: 2022-06-16 18:51:50 +0000 Commit: Alexander Motin CommitDate: 2022-06-16 18:51:50 +0000 amd64: Stop using REP MOVSB for backward memmove()s. Enhanced REP MOVSB feature of CPUs starting from Ivy Bridge makes REP MOVSB the fastest way to copy memory in most of cases. However Intel Optimization Reference Manual says: "setting the DF to force REP MOVSB to copy bytes from high towards low addresses will expe- rience significant performance degradation". Measurements on Intel Cascade Lake and Alder Lake, same as on AMD Zen3 show that it can drop throughput to as low as 2.5-3.5GB/s, comparing to ~10-30GB/s of REP MOVSQ or hand-rolled loop, used for non-ERMS CPUs. This patch keeps ERMS use for forward ordered memory copies, but removes it for backward overlapped moves where it does not work. This is just a cosmetic sync with kernel, since libc does not use ERMS at this time. Reviewed by: mjg MFC after: 2 weeks --- lib/libc/amd64/string/memmove.S | 8 -------- 1 file changed, 8 deletions(-) diff --git a/lib/libc/amd64/string/memmove.S b/lib/libc/amd64/string/memmove.S index 3d75ff45c798..ea92cb18782a 100644 --- a/lib/libc/amd64/string/memmove.S +++ b/lib/libc/amd64/string/memmove.S @@ -274,13 +274,6 @@ __FBSDID("$FreeBSD$"); ALIGN_TEXT 2256: std -.if \erms == 1 - leaq -1(%rdi,%rcx),%rdi - leaq -1(%rsi,%rcx),%rsi - rep - movsb - cld -.else leaq -8(%rdi,%rcx),%rdi leaq -8(%rsi,%rcx),%rsi shrq $3,%rcx @@ -290,7 +283,6 @@ __FBSDID("$FreeBSD$"); movq %rdx,%rcx andb $7,%cl jne 2004b -.endif \end ret .endif