From nobody Thu Jun 30 01:16:32 2022 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 3816C87C171; Thu, 30 Jun 2022 01:16:33 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4LYL5F0V37z4Wbr; Thu, 30 Jun 2022 01:16:33 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1656551793; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=H32yVE9SggyGZ62nmCB6ehnT9qIFqTe0arPsq8OhxCU=; b=qYeBxKhp/FDmTzNlVc9U75tDcqxi3raJhsps7E8wW4IapS79uZs56650hm+B5LBzg4lfTh 79A/jI7zIpTmOfYR0tD9we53Z1Hll4v1GSfg+dtx82+1eF++Z7bLcAQk44CciW6upqhQ7k 205ZCfyRc4HxXbin2m+UXsh91AOBpLDNMEenXQNXqmtoAXVLEWpB2RwOCgThZZ8Bx2IfOR VNSMmGOsSe6ILZnEbghTMMc2NHXG3+wUKfASnu1J0gujeIgxJD96inYcoQ9Xaa6PQwuvfZ 0kGulxMZ55pU2+nCIZXFe9fKqS2anAqa19Dw0xc7MxGCx6FVtNKbnAoL0PsaFA== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id E4D5D1C5E4; Thu, 30 Jun 2022 01:16:32 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 25U1GW5c098104; Thu, 30 Jun 2022 01:16:32 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 25U1GWKO098103; Thu, 30 Jun 2022 01:16:32 GMT (envelope-from git) Date: Thu, 30 Jun 2022 01:16:32 GMT Message-Id: <202206300116.25U1GWKO098103@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Alexander Motin Subject: git: fb07770acdf1 - stable/12 - amd64: Stop using REP MOVSB for backward memmove()s. List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-all@freebsd.org X-BeenThere: dev-commits-src-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: mav X-Git-Repository: src X-Git-Refname: refs/heads/stable/12 X-Git-Reftype: branch X-Git-Commit: fb07770acdf179a609261e09bb895c52522c9d7b Auto-Submitted: auto-generated ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1656551793; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=H32yVE9SggyGZ62nmCB6ehnT9qIFqTe0arPsq8OhxCU=; b=Y2EAJoJ9vr2DtamnE4RdtOItGD+YoHTyoKLrAhWm1Nv49XHN6uV7cgJ3RuK41Pe4K0p//5 iiPghhUTxGvuzx6KD3+HC3lFyybdzhnGLZKAtJ5hgYPDF8RxX41MD07KIb32JIYeB+nTJc sNMoYFjSvcuc2NJuvze4ZFk71EoY+HliPWKoUNYXViG7s3u7OYAob58H2RS2/X7jUqY3lq MfshIeLkKrQLjq/vwBPrIep4qarPrnPAISv4hLpij/gVfmZvERmy7Kc+H6iA3kBOEVh5dx ljpnOpxPmx/XAm3mebNnoQseQplsAC10Ox2hrTu4y1A6TqReVtyMVz7IhpDj9Q== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1656551793; a=rsa-sha256; cv=none; b=jJBeg+PG8vU/15+CrBYY8XbP/bGO14/D9kpdcs3Oj9BuDzHoub3T76OHP+tC2xgz4DsjUs w3sP9HxyRGWOw7xjNDb4x9hxZHrJK8g/QGpYbHyWr9zF93SXtHDoQV0jWKNUUDoSiJ6jJQ 2VaMqZtWgJ4gLyyjRpY42Zpb9LstStOSL/VbWHKQ2fP7+zt8KZLp8wIYqU6BSszVLZMyYB QQ0lP9zPjQ78+Civ2rEPql6owgnRDAkG3vGjLirpAKyr7hWsvpM3+nZR8bdCEfEXpejWQh TWRAlFgJ561uIHV5qetUGkjnD9FGXOCPFn1dTjQ+bvdIRwwLynIsMuagRp+tiQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N The branch stable/12 has been updated by mav: URL: https://cgit.FreeBSD.org/src/commit/?id=fb07770acdf179a609261e09bb895c52522c9d7b commit fb07770acdf179a609261e09bb895c52522c9d7b Author: Alexander Motin AuthorDate: 2022-06-16 17:01:12 +0000 Commit: Alexander Motin CommitDate: 2022-06-30 01:16:03 +0000 amd64: Stop using REP MOVSB for backward memmove()s. Enhanced REP MOVSB feature of CPUs starting from Ivy Bridge makes REP MOVSB the fastest way to copy memory in most of cases. However Intel Optimization Reference Manual says: "setting the DF to force REP MOVSB to copy bytes from high towards low addresses will expe- rience significant performance degradation". Measurements on Intel Cascade Lake and Alder Lake, same as on AMD Zen3 show that it can drop throughput to as low as 2.5-3.5GB/s, comparing to ~10-30GB/s of REP MOVSQ or hand-rolled loop, used for non-ERMS CPUs. This patch keeps ERMS use for forward ordered memory copies, but removes it for backward overlapped moves where it does not work. Reviewed by: mjg MFC after: 2 weeks (cherry picked from commit 6210ac95a19416832601b571409a3e08b76d107f) --- sys/amd64/amd64/support.S | 8 -------- 1 file changed, 8 deletions(-) diff --git a/sys/amd64/amd64/support.S b/sys/amd64/amd64/support.S index f2bc07701f3c..5e1b980c0ea2 100644 --- a/sys/amd64/amd64/support.S +++ b/sys/amd64/amd64/support.S @@ -531,13 +531,6 @@ END(memcmp) ALIGN_TEXT 2256: std -.if \erms == 1 - leaq -1(%rdi,%rcx),%rdi - leaq -1(%rsi,%rcx),%rsi - rep - movsb - cld -.else leaq -8(%rdi,%rcx),%rdi leaq -8(%rsi,%rcx),%rsi shrq $3,%rcx @@ -547,7 +540,6 @@ END(memcmp) movq %rdx,%rcx andb $7,%cl jne 2004b -.endif \end ret .endif