From nobody Mon Apr 10 15:31:28 2023 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PwCcs4hNmz44NkG; Mon, 10 Apr 2023 15:31:41 +0000 (UTC) (envelope-from rick.macklem@gmail.com) Received: from mail-pj1-x1035.google.com (mail-pj1-x1035.google.com [IPv6:2607:f8b0:4864:20::1035]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4PwCcs2nK6z4K7L; Mon, 10 Apr 2023 15:31:41 +0000 (UTC) (envelope-from rick.macklem@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-pj1-x1035.google.com with SMTP id jx2-20020a17090b46c200b002469a9ff94aso3042863pjb.3; Mon, 10 Apr 2023 08:31:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1681140700; x=1683732700; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=vWSOoJlaCa+UfMPHSJkuhgU8wVUshDQkP8e18epaL/I=; b=Shadk/T0HJ1XFjoZFvE/dU5ISfskKqJVJL/hwq5aN+AX1+HW89NKYKDlyKsik0X6X0 DREI1hRo8FWqscF4caq0GWM4oWA8FIGDcU8Jg+dWA6UUNTV/rdBkgtX37a8XP0zRIofo 10A162BalhkFw8hNk/uYCMPzZK63JimGpFoOETjklUdDzyNbW1zNLK2VquZba5IbBJnF 6tPLXNc+oqAr/Ew2GK7zJieyndq0B1jqFZIKyDWPZBLbxYQqxrIeaTZ0drBfDNjA+Dux IZqrMCojULs6Og+xJKiIE8VErHl02VRQnYsSy4qLvJUeaR/GpIkfbmzhx38OcGkAamoj g5Fg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1681140700; x=1683732700; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=vWSOoJlaCa+UfMPHSJkuhgU8wVUshDQkP8e18epaL/I=; b=StvM11HMBjBn8qtAjn6VlXqcWjWkVJtdN4AC/tOtsRGPZylz3wiko4DR1pc7CKmhHM riJqESr9/aPnyk+esuiHNH1xA6lvEoZ25VOz05Ka2Ufn+hSKvkQsHFmg2zqWpeRkQGpL AupLzQ0TSQ/HRDqxhACvMkNLMsZh81sq7Dh3//IZ9cz8S1SPp3+Bo3urt+OKKWc+H6vj 2g6ZGLvoMftICmAcdOw/7tWWICByoM0UtDdmKXUwt8eQEy83QiZ0CiQ9pljK6Dl3fI92 w7U2y3E36ZBqSCq2lwBso1BbLe17DTr4DFaXmk5dTsXw+sIKnmjh1A9XOCjtxY6jSxoX jQ1A== X-Gm-Message-State: AAQBX9duyvCiUVKZC95jAr4TQoTc7ZOau3ML8pekI9tZSiQMDigNozn2 92llAmFc5xjKk7l8NVqilGILAZPO0bD1+8Vqx+UnQN9nOg== X-Google-Smtp-Source: AKy350bcCMcCQrCGOGf2CLIGJVkci3tTgmQcExXRgiUcL/sWIyw1qbzSVjVlPwkcDx5Ggx89sJYMHrFGGRRqg39i91o= X-Received: by 2002:a17:90a:f10a:b0:236:a1f9:9a9d with SMTP id cc10-20020a17090af10a00b00236a1f99a9dmr5946591pjb.2.1681140699854; Mon, 10 Apr 2023 08:31:39 -0700 (PDT) List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-all@freebsd.org X-BeenThere: dev-commits-src-all@freebsd.org MIME-Version: 1.0 References: <202101030102.10312IAC061762@gitrepo.freebsd.org> In-Reply-To: From: Rick Macklem Date: Mon, 10 Apr 2023 08:31:28 -0700 Message-ID: Subject: Re: git: c98a764c681f - main - cp(1): fix performance issue for large non-sparse file copies To: Alexey Dokuchaev Cc: Rick Macklem , src-committers@freebsd.org, dev-commits-src-all@freebsd.org, dev-commits-src-main@freebsd.org Content-Type: multipart/mixed; boundary="0000000000003ba28b05f8fd12ea" X-Rspamd-Queue-Id: 4PwCcs2nK6z4K7L X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; TAGGED_FROM(0.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N --0000000000003ba28b05f8fd12ea Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Mon, Apr 10, 2023 at 12:54=E2=80=AFAM Alexey Dokuchaev wrote: > > CAUTION: This email originated from outside of the University of Guelph. = Do not click links or open attachments unless you recognize the sender and = know the content is safe. If in doubt, forward suspicious emails to IThelp@= uoguelph.ca > > > On Sun, Jan 03, 2021 at 01:02:18AM +0000, Rick Macklem wrote: > > commit c98a764c681f8b70812a9f13a6e61c96aa1a69d2 > > > > cp(1): fix performance issue for large non-sparse file copies > > > > PR252358 reported a serious performance problem when > > copying a large non-sparse file on a UFS file system. > > This problem seems to have been caused by a large > > number of SEEK_HOLE operations, with one done > > for each copy_file_range(2) call. > > > > This patch modifies cp(1) to use a large (SSIZE_MAX) > > len argument, reducing the number of system calls > > and resolving the performance issue. > > char *p; > > > > @@ -236,7 +235,7 @@ copy_file(const FTSENT *entp, int dne) > > do { > > if (use_copy_file_range) { > > rcount =3D copy_file_range(from_fd, NULL, > > - to_fd, NULL, bufsize, 0); > > + to_fd, NULL, SSIZE_MAX, 0); > > Hi Rick, > > This change unfortunately breaks copying files in resource-limited > environments (e.g. many port builders do that to prevent runaway > processes): > > ulimit -f 16384000 > cp -p packages/13.0-i386-wip/All/perl5-5.32.1_3.tbz /tmp ; echo $? > Filesize limit exceeded > 153 > > Previously bufsize was 2097152 which was a lot saner than current > 9223372036854775807. Perhaps we should set it per getrlimit(2) > value for RLIMIT_FSIZE? I think zfs_copy_file_range() needs to use vn_rlimit_fsizex() the same way that vn_generic_copy_file_range() does. I have posted the attached patch to D39419. danfe@. Assuming you were using zfs, could you test this patch? (You will need an up to date main kernel and, hopefully, the block cloning stuff has not trashed your zpool.) rick > > ./danfe > --0000000000003ba28b05f8fd12ea Content-Type: application/octet-stream; name="zfscopyrlimit.patch" Content-Disposition: attachment; filename="zfscopyrlimit.patch" Content-Transfer-Encoding: base64 Content-ID: X-Attachment-Id: f_lgazr7f60 LS0tIHpmc192bm9wc19vcy5jLnNhdgkyMDIzLTA0LTEwIDA4OjAxOjA1LjkwNTkwNjAwMCAtMDcw MAorKysgemZzX3Zub3BzX29zLmMJMjAyMy0wNC0xMCAwODoxNDo0MC41NjM3NDMwMDAgLTA3MDAK QEAgLTYyNDIsNyArNjI0Miw4IEBAIHpmc19mcmVlYnNkX2NvcHlfZmlsZV9yYW5nZShzdHJ1Y3Qg dm9wX2NvcHlfZmlsZV9yYW5nZQogCXN0cnVjdCBtb3VudCAqbXA7CiAJc3RydWN0IHVpbyBpbzsK IAlpbnQgZXJyb3I7Ci0JdWludDY0X3QgbGVuID0gKmFwLT5hX2xlbnA7CisJdWludDY0X3QgbGVu OworCXNzaXplX3QgciA9IDA7CiAKIAkvKgogCSAqIFRPRE86IElmIG9mZnNldC9sZW5ndGggaXMg bm90IGFsaWduZWQgdG8gcmVjb3Jkc2l6ZSwgdXNlCkBAIC02MjgwLDkgKzYyODEsMTQgQEAgemZz X2ZyZWVic2RfY29weV9maWxlX3JhbmdlKHN0cnVjdCB2b3BfY29weV9maWxlX3JhbmdlCiAKIAlp by51aW9fb2Zmc2V0ID0gKmFwLT5hX291dG9mZnA7CiAJaW8udWlvX3Jlc2lkID0gKmFwLT5hX2xl bnA7Ci0JZXJyb3IgPSB2bl9ybGltaXRfZnNpemUob3V0dnAsICZpbywgYXAtPmFfZnNpemV0ZCk7 CisJZXJyb3IgPSB2bl9ybGltaXRfZnNpemV4KG91dHZwLCAmaW8sIDAsICZyLCBhcC0+YV9mc2l6 ZV90ZCk7CiAJaWYgKGVycm9yICE9IDApCiAJCWdvdG8gb3V0X2xvY2tlZDsKKwlsZW4gPSBpby51 aW9fcmVzaWQ7CisJLyoKKwkgKiBObyBuZWVkIHRvIGNhbGwgdm5fcmxpbWl0X2ZzaXpleF9yZXMg YmVmb3JlIHJldHVybiwKKwkgKiBzaW5jZSB0aGUgdWlvIGlzIGxvY2FsLgorCSAqLwogCiAJZXJy b3IgPSB6ZnNfY2xvbmVfcmFuZ2UoVlRPWihpbnZwKSwgYXAtPmFfaW5vZmZwLCBWVE9aKG91dHZw KSwKIAkgICAgYXAtPmFfb3V0b2ZmcCwgJmxlbiwgYXAtPmFfb3V0Y3JlZCk7Cg== --0000000000003ba28b05f8fd12ea--