From nobody Thu Feb 23 21:02:04 2023 X-Original-To: freebsd-fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PN57Y4TtPz3sljt for ; Thu, 23 Feb 2023 21:02:17 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: from mail-yw1-x1131.google.com (mail-yw1-x1131.google.com [IPv6:2607:f8b0:4864:20::1131]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4PN57Y44JHz3FyY for ; Thu, 23 Feb 2023 21:02:17 +0000 (UTC) (envelope-from fjwcash@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-yw1-x1131.google.com with SMTP id 00721157ae682-536c02eea4dso217087537b3.4 for ; Thu, 23 Feb 2023 13:02:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=sohsxX2y8Z7PSAgmLhhEfuPmQPSXhtFYlLhRm57KNlA=; b=LbpjNyPHyaVAqQcpQEb4Ql/VEKN77ooNkKfNHzg2wdbNghnkTSalkgntGCCvAr+iAE 5yWKXXezHoThdLsQ8ZH3kVSv3u2/AEm0hDfEcfOEV+YoHdD3m8hh913qqxV6cjMohVCQ +wqmqi1lcjjWKlIS9YU9m/UYo6fGgf+iiJy4az18yRXyOpg74AqBjRwQULx/WVGln1Kd InNa8ffblQDfOz9roPQbCuiPSbTEvWqeOZ+RKCXnyrTG3/Btjb8nUmo8Zq/on3/Hag4C 3CqOX7njmaI4dx6oCowzSUvbQ5LgJ1aDP/lv+7CCfiS5fhYiMv3uFgwfoF9aSOZXoJIZ NTew== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=sohsxX2y8Z7PSAgmLhhEfuPmQPSXhtFYlLhRm57KNlA=; b=BcaI5+2bLhZdT2lxOXbp0YUUo4kgsasLwaIHcf+WrXQc2Uav1+C03LJATSTb80oFl4 8SgJn0E9PKnvlwG/HrgPN8goGcoUhyyAU9XT2XORq6xyMKyEH1id0DuwUnnSCAcQKfep R4nr1C6BIGt2AO62hKJFJw8YYzxT7K0mEyedz0als7CUVFly030VytVoq0ej0gaS1Jyl eQ+AWqagYaRfateYtVkm8bEsy8bfl8mm9YU6fdZ/7ApkYBNmFK9VVVrGbfndlYqoDCKZ lmNTTv3FDLv5TSOJFyoikSKbUo8Scl4Itfh3YxJlkZs4D9el3rAlx7BVJOOyqDEsweZO OaFA== X-Gm-Message-State: AO0yUKXYzUvc/2uaMZwLsm+HFTBI1fn5583/8bWYgPnu5UKDdKJ+pxY5 8KxuwGYdGb4v/JiySjYrMACeiJl/7OuPSxcqW0o= X-Google-Smtp-Source: AK7set8PfdYKPOZRdIaU85+Ce9CX90cELcsgPr2CBuiXYbAoiMHo31uhax98YG5VHxKTxZEQahyZkIYglcKyV7KovXM= X-Received: by 2002:a25:9392:0:b0:997:bdfe:78c5 with SMTP id a18-20020a259392000000b00997bdfe78c5mr2756849ybm.6.1677186135972; Thu, 23 Feb 2023 13:02:15 -0800 (PST) List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org MIME-Version: 1.0 References: <866d6937-a4e8-bec3-d61b-07df3065fca9@sentex.net> <1031e2b0-b245-1dc6-a499-8f4da3796543@quip.cz> <46455168-d7f1-6ca9-ad2f-9bcd3359e0f3@sentex.net> <78c78aec-a34b-f188-ef96-8ced9a1eda35@quip.cz> <741387429.91447.1677122934622@ichabod.co-bxl> In-Reply-To: From: Freddie Cash Date: Thu, 23 Feb 2023 13:02:04 -0800 Message-ID: Subject: Re: speeding up zfs send | recv (update) To: Chris Watson Cc: Sysadmin Lists , freebsd-fs Content-Type: multipart/alternative; boundary="000000000000db5f5d05f5645322" X-Rspamd-Queue-Id: 4PN57Y44JHz3FyY X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N --000000000000db5f5d05f5645322 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Thu, Feb 23, 2023 at 11:15 AM Chris Watson wrote: > [Sorry miroslav, I hit send without checking the To: this was meant to be > public] > > I=E2=80=99m a bit late, but I mentioned this to someone on this thread pr= ivately, > I=E2=80=99m curious why =E2=80=98spiped=E2=80=99 hasn=E2=80=99t been ment= ioned in this thread. I=E2=80=99ve seen > everything from VPN=E2=80=99s to nc. VPNs would be, imo, grossly > unwarranted/massively overly complex/hard to secure just to simply have a > secure pipe for doing ZFS send|recv. > > Simply configuring an spiped PtP pipe between A and B seems the simplest, > most secure, performant option here. At least considering all the other > options tossed out in this thread. > > No one=E2=80=99s using spiped? O.o > > Thoughts? > > Has anyone compared ssh to spiped regarding overhead and throughput in > this scenario? > Interesting, never heard of spiped before this message. Looks neat. Not really useful for our specific use case (sending ZFS snapshots across private LAN and WAN links without encryption). But could be useful in place of SSH connections when sending across public links, depending on cipher suites used. --=20 Freddie Cash fjwcash@gmail.com --000000000000db5f5d05f5645322 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
On Thu, Feb 23, 2023 at 11:15 AM Chris Wa= tson <bsdunix44@gmail.com>= wrote:
[Sorry miroslav, I = hit send without checking the To: this was meant to be public]=C2=A0
<= div dir=3D"auto" style=3D"font-size:1rem;word-spacing:1px;border-color:rgb(= 49,49,49);color:rgb(49,49,49)">
I= =E2=80=99m a bit late, but I mentioned this to someone on this thread priva= tely, I=E2=80=99m curious why =E2=80=98spiped=E2=80=99 hasn=E2=80=99t been = mentioned in this thread. I=E2=80=99ve seen everything from VPN=E2=80=99s t= o nc. VPNs would be, imo, grossly unwarranted/massively overly complex/hard= to secure just to simply have a secure pipe for doing ZFS send|recv.=C2=A0=

Simply con= figuring an spiped PtP pipe between A and B seems the simplest, most secure= , performant option here. At least considering all the other options tossed= out in this thread.=C2=A0

No one=E2=80=99s using spiped? O.o
Thoughts?=C2=A0

Has anyone compared ssh to spi= ped regarding overhead and throughput in this scenario?

Interesting, never heard of spiped before this mes= sage.=C2=A0 Looks neat.

Not really useful for our = specific use case (sending ZFS snapshots across private LAN and WAN links w= ithout encryption).=C2=A0 But could be useful in place of SSH connections w= hen sending across public links, depending on cipher suites used.

--
Freddie = Cash
fjwcash@gmai= l.com
--000000000000db5f5d05f5645322--