Re: can sftp be made multi-threaded?

On Wed, Aug 09, 2023 at 02:49:48PM +0200, Juraj Lutter wrote:

>Have you played with misc/mbuffer from ports? It might do what you are 
>looking for (provided that sftp alone isn’t sufficient).

I've used mbuffer in a remote zfs snapshot context before but
I thought that was to account the bursty nature of sending zfs backups
because reading from the disks was bursty. This isn't the case here,
as disks are all m.2 nvme connected to pci4 risers.

The problem I'm seeing I can see through a tool like
speedtest-go and running the test in single-thread mode. If 10 threads
are used, I can saturate the connection. If 1 thread is used, the connection
speed reported is a small fraction of the actual. I'm hypothesising that
throughput-per-thread might be limited either by the ISPs and/or by
for example sysctls on the two freebsd systems.

I know there's something like third-party patches like HPN(?) that *might*
do what I want but for a start I'm not sure they will work with openssh v3.

I'm interested how others accomplish secure backup over long distances
at near-line-speeds. If they need to recompile base openssh to accomplish it,
if they're using v3 and any sysctls or other settings in ssh_/sshd_config
--