From nobody Sun Nov 13 19:03:15 2022 X-Original-To: freebsd-fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4N9ML06MQpz4dHCr for ; Sun, 13 Nov 2022 19:03:52 +0000 (UTC) (envelope-from m.e.sanliturk@gmail.com) Received: from mail-yb1-xb2b.google.com (mail-yb1-xb2b.google.com [IPv6:2607:f8b0:4864:20::b2b]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4N9ML04T8bz438f for ; Sun, 13 Nov 2022 19:03:52 +0000 (UTC) (envelope-from m.e.sanliturk@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-yb1-xb2b.google.com with SMTP id f201so10000738yba.12 for ; Sun, 13 Nov 2022 11:03:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=XjKXLVDOaCfKDEn/udFPGSAmTCJSduQmTy7N/klTUPY=; b=I5ygKeeITB+FcE2qtyib7TA4/7wRzQa0XPRuFt2SlGuWf7p112m4ViYOtE/p+wW0BX bVU3VZLeVxzvJu1ki0wL3+k3QatNmfWTmBvb9EsRNs3Q5xhlKa+JuLQQw36JkRFvfbPQ Uwh/wYcHFoSohCVVN0XkmpS6zyt6FyNFjaFmnxtQbkCbKwEU+xDUa8vHgdRJ3TkCWKQu 5VdEbqcZTrxbGDni17XUgIu/Zs5I+yZnnpBVylPFfeeUflh7V+fbCY/2EwvRS3xvT/H3 peMBSrfHWM409p3xzaqEfmu/iUDI6mjmnE4PptBY3nT604n8bsbkatVby0lfSOOq0W8q fwgQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=XjKXLVDOaCfKDEn/udFPGSAmTCJSduQmTy7N/klTUPY=; b=oYyHrtzB9R4mPycXoV+WdTT56S2KKpV041l07a5rv5KJrcNwaBES/dc7VGg9c6yRt2 JY29W33jesSg75l6MpH0ZPRE7Vtd0ZnPjylwKdK90nqXdH13EeHyJMYZhCOcAi4HnrSQ HrELdSmmSpfGzoqcWPCY18QseJV/BTtNF4ULBOm7uOTgWaKuuKmyF2i7lymicyt6uE60 2P4W4dfEZADGm9IuYt6AeirV82ct8rmUzGN17foxCrPG1N5wqlEN3aTP8M4+EKCDiNYn SBsqzS630HT5dgjAqfesNXZe3yGf6OJbSR/mK5/gLTxkTkwqAV4J2mxm1YMgPPdKhjwt 6QXQ== X-Gm-Message-State: ANoB5pnJxV8idclbEdYonxYlp0zHvDL4Bn6YOU2N0gWoyryyCi+OUA1s kQJmmK6WrJWfI6gRYSIfcNkq/YxM4YrqUcVx8mwzkrfiSHk= X-Google-Smtp-Source: AA0mqf64h6njLM0prYKtyeV9DqjMMgXYokecPa4SBNl9vhByfc0728T2SelX4kZ+s8DuDZYNVRjQIIDTiolE62SqM54= X-Received: by 2002:a25:cad4:0:b0:6ca:aadc:c4d2 with SMTP id a203-20020a25cad4000000b006caaadcc4d2mr9996348ybg.77.1668366231846; Sun, 13 Nov 2022 11:03:51 -0800 (PST) List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org MIME-Version: 1.0 References: In-Reply-To: From: Mehmet Erol Sanliturk Date: Sun, 13 Nov 2022 22:03:15 +0300 Message-ID: Subject: Re: Odd behaviour of two identical ZFS servers mirroring via rsync To: Bob Friesenhahn Cc: Mark Saad , freebsd-fs@freebsd.org Content-Type: multipart/alternative; boundary="0000000000009ac45d05ed5ec8bc" X-Rspamd-Queue-Id: 4N9ML04T8bz438f X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; TAGGED_FROM(0.00)[] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N --0000000000009ac45d05ed5ec8bc Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Sun, Nov 13, 2022 at 8:46 PM Bob Friesenhahn < bfriesen@simple.dallas.tx.us> wrote: > On Sun, 13 Nov 2022, Mark Saad wrote: > >> > > Bob are you saying when the target is zfs --inplace --no-whole-file hel= ps > > or just in general when you have > > large files ? Also have you tried using --delete-during / > --delete-after ? > > The '-inplace --no-whole-file' updates the file blocks if they have > changed (comparing the orgin blocks with the existing mirror blocks) > rather than creating a new copy of the file and moving it into place > when it is complete. ZFS does not check if data content has been > changed while it is being written so a write of the same data will > result in a fresh allocation based on its Copy On Write ("COW") > design. Writing a whole new file obviously significantly increases > the number of blocks which are written. Requesting that rsync only > write to the file for the blocks which have changed reduces the total > number of blocks which get written. > > The above helps quite a lot when using snapshots since then fewer > blocks are in the snapshots. > > I have never tried --delete-during so I can't comment on that. > > Bob > -- > Bob Friesenhahn > bfriesen@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen= / > GraphicsMagick Maintainer, http://www.GraphicsMagick.org/ > Public Key, http://www.simplesystems.org/users/bfriesen/public-key.tx= t > > More information on zfs and rsync : https://www.google.com/search?q=3Duse+of+rsync+command+for+zfs+volumes&sxsr= f=3DALiCzsZALxJWD--EizNVNAHl-u-j6BOndw%3A1668365903483&source=3Dhp&ei=3DTz5= xY6XbGrCQxc8Plbyl8A0&iflsig=3DAJiK0e8AAAAAY3FMX56shwXahupzkNcfFLLhJJd0xL1U&= ved=3D0ahUKEwjlvp6o66v7AhUwSPEDHRVeCd4Q4dUDCAc&uact=3D5&oq=3Duse+of+rsync+c= ommand+for+zfs+volumes&gs_lcp=3DCgdnd3Mtd2l6EAMyBQgAEKIEOgQIIxAnOgsILhCABBD= HARDRAzoFCAAQgAQ6CAguENQCEIAEOggILhCABBDUAjoFCC4QgAQ6CAgAEIAEEMsBOggILhCABB= DLAToGCAAQFhAeOggIABAWEB4QDzoFCAAQhgM6BQghEKABOgQIIRAVOggIIRAWEB4QHToHCCEQo= AEQClAAWI6bAmCtnQJoAHAAeACAAbkCiAHkLJIBCDAuMzUuMC4xmAEAoAEB&sclient=3Dgws-w= iz use of rsync command for zfs volumes Mehmet Erol Sanliturk --0000000000009ac45d05ed5ec8bc Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


On Sun, Nov 13, 2022= at 8:46 PM Bob Friesenhahn <bfriesen@simple.dallas.tx.us> wrote:
On Sun, 13 Nov 2022, Mark Saad wrote:
>>
> Bob are you saying when the target is zfs --inplace --no-whole-file he= lps
> or just in general when you have
> large files ?=C2=A0 Also have you tried using --delete-during / --dele= te-after=C2=A0 ?

The '-inplace --no-whole-file' updates the file blocks if they have=
changed (comparing the orgin blocks with the existing mirror blocks)
rather than creating a new copy of the file and moving it into place
when it is complete.=C2=A0 ZFS does not check if data content has been
changed while it is being written so a write of the same data will
result in a fresh allocation based on its Copy On Write ("COW") <= br> design.=C2=A0 Writing a whole new file obviously significantly increases the number of blocks which are written.=C2=A0 Requesting that rsync only write to the file for the blocks which have changed reduces the total
number of blocks which get written.

The above helps quite a lot when using snapshots since then fewer
blocks are in the snapshots.

I have never tried --delete-during so I can't comment on that.

Bob
--
Bob Friesenhahn
bfriesen@= simple.dallas.tx.us, http://www.simplesystems.org/us= ers/bfriesen/
GraphicsMagick Maintainer,=C2=A0 =C2=A0 http://www.GraphicsMagick.org/=
Public Key,=C2=A0 =C2=A0 =C2=A0http://www.= simplesystems.org/users/bfriesen/public-key.txt




More information on zfs and rsync :


https://www.google.com/sea= rch?q=3Duse+of+rsync+command+for+zfs+volumes&sxsrf=3DALiCzsZALxJWD--Eiz= NVNAHl-u-j6BOndw%3A1668365903483&source=3Dhp&ei=3DTz5xY6XbGrCQxc8Pl= byl8A0&iflsig=3DAJiK0e8AAAAAY3FMX56shwXahupzkNcfFLLhJJd0xL1U&ved=3D= 0ahUKEwjlvp6o66v7AhUwSPEDHRVeCd4Q4dUDCAc&uact=3D5&oq=3Duse+of+rsync= +command+for+zfs+volumes&gs_lcp=3DCgdnd3Mtd2l6EAMyBQgAEKIEOgQIIxAnOgsIL= hCABBDHARDRAzoFCAAQgAQ6CAguENQCEIAEOggILhCABBDUAjoFCC4QgAQ6CAgAEIAEEMsBOggI= LhCABBDLAToGCAAQFhAeOggIABAWEB4QDzoFCAAQhgM6BQghEKABOgQIIRAVOggIIRAWEB4QHTo= HCCEQoAEQClAAWI6bAmCtnQJoAHAAeACAAbkCiAHkLJIBCDAuMzUuMC4xmAEAoAEB&sclie= nt=3Dgws-wiz

use of rsync command for zfs volu= mes



Mehmet= Erol Sanliturk





=C2=A0
--0000000000009ac45d05ed5ec8bc--