From nobody Tue Sep 14 21:21:37 2021 X-Original-To: freebsd-geom@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4F94917CFACC for ; Tue, 14 Sep 2021 21:21:47 +0000 (UTC) (envelope-from jmg@gold.funkthat.com) Received: from gold.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "gate2.funkthat.com", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4H8GWG1Nt5z3hkm for ; Tue, 14 Sep 2021 21:21:45 +0000 (UTC) (envelope-from jmg@gold.funkthat.com) Received: from gold.funkthat.com (localhost [127.0.0.1]) by gold.funkthat.com (8.15.2/8.15.2) with ESMTPS id 18ELLbeH060515 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Tue, 14 Sep 2021 14:21:37 -0700 (PDT) (envelope-from jmg@gold.funkthat.com) Received: (from jmg@localhost) by gold.funkthat.com (8.15.2/8.15.2/Submit) id 18ELLbcD060514; Tue, 14 Sep 2021 14:21:37 -0700 (PDT) (envelope-from jmg) Date: Tue, 14 Sep 2021 14:21:37 -0700 From: John-Mark Gurney To: Johannes Totz Cc: freebsd-geom@FreeBSD.org Subject: Re: RFC reviews for ggate and hastd Message-ID: <20210914212137.GI5928@funkthat.com> Mail-Followup-To: Johannes Totz , freebsd-geom@FreeBSD.org References: <20210909223355.GE5928@funkthat.com> <31ac6be7-e438-00d0-b63a-b27b09ed5d58@bruelltuete.com> List-Id: GEOM-specific discussions and implementations List-Archive: https://lists.freebsd.org/archives/freebsd-geom List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-geom@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <31ac6be7-e438-00d0-b63a-b27b09ed5d58@bruelltuete.com> X-Operating-System: FreeBSD 11.3-STABLE amd64 X-PGP-Fingerprint: D87A 235F FB71 1F3F 55B7 ED9B D5FF 5A51 C0AC 3D65 X-Files: The truth is out there X-URL: https://www.funkthat.com/ X-Resume: https://www.funkthat.com/~jmg/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? User-Agent: Mutt/1.6.1 (2016-04-27) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (gold.funkthat.com [127.0.0.1]); Tue, 14 Sep 2021 14:21:37 -0700 (PDT) X-Rspamd-Queue-Id: 4H8GWG1Nt5z3hkm X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of jmg@gold.funkthat.com has no SPF policy when checking 208.87.223.18) smtp.mailfrom=jmg@gold.funkthat.com X-Spamd-Result: default: False [-1.80 / 15.00]; RCVD_TLS_ALL(0.00)[]; ARC_NA(0.00)[]; FREEFALL_USER(0.00)[jmg]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; MID_RHS_MATCH_FROM(0.00)[]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[funkthat.com]; AUTH_NA(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_HAM_SHORT(-1.00)[-1.000]; RCPT_COUNT_TWO(0.00)[2]; R_SPF_NA(0.00)[no SPF record]; FORGED_SENDER(0.30)[jmg@funkthat.com,jmg@gold.funkthat.com]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2]; ASN(0.00)[asn:32354, ipnet:208.87.216.0/21, country:US]; FROM_NEQ_ENVFROM(0.00)[jmg@funkthat.com,jmg@gold.funkthat.com] X-ThisMailContainsUnwantedMimeParts: N Johannes Totz wrote this message on Mon, Sep 13, 2021 at 02:00 +0100: > On 09/09/2021 23:33, John-Mark Gurney wrote: > > Johannes Totz via freebsd-geom wrote this message on Thu, Sep 02, 2021 at 21:55 +0100: > >> (looks like gmane swallowed my 1st message, trying again) > >> > >> Hey folks, > >> > >> any ggate or hastd users here? I've got some code reviews for you. > >> Please take a look if you get a chance: > >> > >> https://reviews.freebsd.org/D31727 > >> Fix potential out-of-bounds read in the geom-gate kernel module. > >> > >> https://reviews.freebsd.org/D31722 > >> Dynamically alloc buffers in ggatec, instead of assuming a fixed size on > >> the stack. > >> > >> https://reviews.freebsd.org/D31709 > >> Simple rc script to start ggated. > > > > I'll try to look at them. > > > > I've broken out the ggate code to: https://www.funkthat.com/gitea/jmg/ggate > > Nice, thanks! > > I noticed the http branch. One weekend toy project idea I wanted to get > around to was to write a ggated impl that talks to Backblaze. I looked at the Backblaze B2 API, and I don't see a way to do partial updates of a file. All the API that I see require you to upload the entire file, so I don't think that it'll work. I abanded http as a solution, because of issues w/ WebDAV and partial updates and the IETF not being very sane about it: https://blog.sphere.chronosempire.org.uk/2012/11/21/webdav-and-the-http-patch-nightmare Also, the fact that apache would randomly truncate a file didn't help either... > > and have a number of fixes in that tree, one of them being having the > > server (ggated) be multi-threaded, to improve IOPs. Another is dropping > > the setting of tcp buffer sizes to allow the dynamic buffer sizing code > > to work, allowing it to grow larger than 128kB for better perf on high > > latency links.. > > > > In another branch, I have a version that runs the client over ssh using > > sftp (requires a modified sftp-server for best performance, and security) > > allowing for better security.. > > > > The ssh branch does have an rc.d script for that. I'll look at which one > > should be used, and adapting the man page that I've written for that one > > as well. > > > > Thanks for the help/work! -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not."