From nobody Tue Jul 16 21:40:26 2024
X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
	by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WNsv84LbMz5QJ43
	for <freebsd-stable@mlmmj.nyi.freebsd.org>; Tue, 16 Jul 2024 21:40:52 +0000 (UTC)
	(envelope-from jason@tubnor.net)
Received: from mail.tubnor.net (mail.tubnor.net [103.236.162.16])
	(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
	 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA512)
	(Client did not present a certificate)
	by mx1.freebsd.org (Postfix) with ESMTPS id 4WNsv654D7z4KyH
	for <freebsd-stable@freebsd.org>; Tue, 16 Jul 2024 21:40:50 +0000 (UTC)
	(envelope-from jason@tubnor.net)
Authentication-Results: mx1.freebsd.org;
	dkim=pass header.d=tubnor.net header.s=20220915 header.b=lKjqGhBT;
	dmarc=pass (policy=none) header.from=tubnor.net;
	spf=pass (mx1.freebsd.org: domain of jason@tubnor.net designates 103.236.162.16 as permitted sender) smtp.mailfrom=jason@tubnor.net
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tubnor.net; s=20220915;
	t=1721166038;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=Adjx/0eIfYXGRqG/bVa0M/MmugNfP+CAP84CgiP+Ra8=;
	b=lKjqGhBTUhsiB0PBxABxH19sasfh7vrlNeCGbkSXwXxLIkcfvszj8ybLXQTZTlotqLV6e+
	9ELX9wFuLNj3AvGgi3lrnTSVIVamGWDesbgacRgkgu7+xMxNdYW7/CdyyaJyhTK8TkoV5I
	ef1ODNeI/YeR+MKLKFVdHN+hF9qxwFI=
Received: from smtpclient.apple (<unknown> [2001:8004:44b0:1b09:c528:56e8:c3ea:9263])
	by mel01.ar18.net (OpenSMTPD) with ESMTPSA id 2af21a72 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO);
	Wed, 17 Jul 2024 07:40:37 +1000 (AEST)
Content-Type: multipart/alternative; boundary=Apple-Mail-E7688D7A-DC31-4F7B-96F1-10E612C87F8B
Content-Transfer-Encoding: 7bit
From: Jason Tubnor <jason@tubnor.net>
List-Id: Production branch of FreeBSD source code <freebsd-stable.freebsd.org>
List-Archive: https://lists.freebsd.org/archives/freebsd-stable
List-Help: <mailto:stable+help@freebsd.org>
List-Post: <mailto:stable@freebsd.org>
List-Subscribe: <mailto:stable+subscribe@freebsd.org>
List-Unsubscribe: <mailto:stable+unsubscribe@freebsd.org>
X-BeenThere: freebsd-stable@freebsd.org
Sender: owner-freebsd-stable@FreeBSD.org
Mime-Version: 1.0 (1.0)
Subject: Re: Possible bug in zfs send or pipe implementation?
Date: Wed, 17 Jul 2024 07:40:26 +1000
Message-Id: <F34D0C07-E384-418C-B8EE-111AE094F4A1@tubnor.net>
References: <b8d25aa4-4f66-440e-b2ba-043c4dcfed69@tubnor.net>
Cc: freebsd-stable@freebsd.org
In-Reply-To: <b8d25aa4-4f66-440e-b2ba-043c4dcfed69@tubnor.net>
To: Garrett Wollman <wollman@bimajority.org>,
 Rick Macklem <rick.macklem@gmail.com>
X-Mailer: iPhone Mail (21F90)
X-Spamd-Bar: --
X-Spamd-Result: default: False [-3.00 / 15.00];
	SUBJECT_ENDS_QUESTION(1.00)[];
	NEURAL_HAM_MEDIUM(-1.00)[-1.000];
	NEURAL_HAM_LONG(-1.00)[-1.000];
	NEURAL_HAM_SHORT(-1.00)[-1.000];
	DMARC_POLICY_ALLOW(-0.50)[tubnor.net,none];
	R_DKIM_ALLOW(-0.20)[tubnor.net:s=20220915];
	R_SPF_ALLOW(-0.20)[+ip4:103.236.162.16];
	MIME_GOOD(-0.10)[multipart/alternative,text/plain];
	RCVD_TLS_ALL(0.00)[];
	DKIM_TRACE(0.00)[tubnor.net:+];
	FROM_HAS_DN(0.00)[];
	FREEMAIL_TO(0.00)[bimajority.org,gmail.com];
	RCPT_COUNT_THREE(0.00)[3];
	ARC_NA(0.00)[];
	RCVD_COUNT_ONE(0.00)[1];
	APPLE_IOS_MAILER_COMMON(0.00)[];
	ASN(0.00)[asn:133159, ipnet:103.236.162.0/23, country:AU];
	MLMMJ_DEST(0.00)[freebsd-stable@freebsd.org];
	TO_MATCH_ENVRCPT_SOME(0.00)[];
	FROM_EQ_ENVFROM(0.00)[];
	FREEFALL_USER(0.00)[jason];
	MID_RHS_MATCH_FROM(0.00)[];
	RCVD_VIA_SMTP_AUTH(0.00)[];
	TAGGED_RCPT(0.00)[];
	MIME_TRACE(0.00)[0:+,1:+,2:~];
	TO_DN_SOME(0.00)[]
X-Rspamd-Queue-Id: 4WNsv654D7z4KyH


--Apple-Mail-E7688D7A-DC31-4F7B-96F1-10E612C87F8B
Content-Type: text/plain;
	charset=utf-8
Content-Transfer-Encoding: quoted-printable

Confirmed. Removing PV from the pipeline has solved my issue. I was able to m=
ove 20TB around in the last 18 hours without issue.

Sent from my iPhone

> On 15 Jul 2024, at 6:42=E2=80=AFPM, Jason Tubnor <jason@tubnor.net> wrote:=

>=20
> =EF=BB=BF
>=20
>=20
> On 15/07/2024 9:35 am, Garrett Wollman wrote:
>> I'm currently running syncoid with `--quiet`, which removes `pv` from
>> the pipeline.  It seems to be moving along, but of course I can't
>> easily tell what it's doing.
>>=20
>> Without `pv` in the way, the process reading the pipe is `lzop`
>> instead, which doesn't try to do any fancy select() stuff, it's just a
>> normal filter reading data from stdin and writing compressed data to
>> stdout, in sequence.
> I'm having the same issue here. I thought it was just my hardware and/or s=
yncoid that was having this issue on 3.5TB dataset replications.
>=20
> I'll try the quiet mode to see if removing PV solves the problem. I'm also=
 using compression=3Dnone so I won't see lzop.
>=20
> Cheers.
>=20
>>=20

--Apple-Mail-E7688D7A-DC31-4F7B-96F1-10E612C87F8B
Content-Type: text/html;
	charset=utf-8
Content-Transfer-Encoding: quoted-printable

<html><head><meta http-equiv=3D"content-type" content=3D"text/html; charset=3D=
utf-8"></head><body dir=3D"auto">Confirmed. Removing PV from the pipeline ha=
s solved my issue. I was able to move 20TB around in the last 18 hours witho=
ut issue.<div><br id=3D"lineBreakAtBeginningOfSignature"><div dir=3D"ltr">Se=
nt from my iPhone</div><div dir=3D"ltr"><br><blockquote type=3D"cite">On 15 J=
ul 2024, at 6:42=E2=80=AFPM, Jason Tubnor &lt;jason@tubnor.net&gt; wrote:<br=
><br></blockquote></div><blockquote type=3D"cite"><div dir=3D"ltr">=EF=BB=BF=


 =20
    <meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3DUTF-8"=
>
 =20
 =20
    <p><br>
    </p>
    <div class=3D"moz-cite-prefix">On 15/07/2024 9:35 am, Garrett Wollman
      wrote:<br>
    </div>
    <blockquote type=3D"cite" cite=3D"mid:26260.24797.484878.694868@hergotha=
.csail.mit.edu">
      <pre class=3D"moz-quote-pre" wrap=3D"">I'm currently running syncoid w=
ith `--quiet`, which removes `pv` from
the pipeline.  It seems to be moving along, but of course I can't
easily tell what it's doing.

Without `pv` in the way, the process reading the pipe is `lzop`
instead, which doesn't try to do any fancy select() stuff, it's just a
normal filter reading data from stdin and writing compressed data to
stdout, in sequence.
</pre>
    </blockquote>
    <p>I'm having the same issue here. I thought it was just my hardware
      and/or syncoid that was having this issue on 3.5TB dataset
      replications.</p>
    <p>I'll try the quiet mode to see if removing PV solves the problem.
      I'm also using compression=3Dnone so I won't see lzop.</p>
    <p>Cheers.<br>
    </p>
    <blockquote type=3D"cite" cite=3D"mid:26260.24797.484878.694868@hergotha=
.csail.mit.edu">
      <pre class=3D"moz-quote-pre" wrap=3D""><span style=3D"white-space: pre=
-wrap">
</span></pre>
    </blockquote>
 =20

</div></blockquote></div></body></html>=

--Apple-Mail-E7688D7A-DC31-4F7B-96F1-10E612C87F8B--