Re: can sftp be made multi-threaded?

On Fri, Aug 11, 2023 at 12:21:33PM +0100, Jamie Landeg-Jones wrote:

>rsync just spawns an ssh command, so would probably behave similarly.

I'm hoping that rsync will spawn many ssh. Need to look at max sessions 
on both ends of the connection.

Since encountering the described problem, the person at the other
end is away for the week so have not been able to test thoroughly.
What I have been able to test shows that there is spiky latency
in the connection, as well as slow speed, single-threaded.

>Another thing, scp transfers from my test Rpi2 are much slower than the network
>can handle due to the CPU use, which hits 100% on one cpu whilst it's running.
>So, check that CPU isn't the bottleneck too.

Yup. That won't be happening here. Dual xenon with 56 cores at remote
end and same (but with 32 cores) at this end

>As for the speed, I just tested sftp to transfer a file of random data, 2 GB in
>size from one FreeBSD box in London to another in France:
>
>The final result was:
>
> 100% 2000MB  43.5MB/s   00:46  (Note, that's MegaBYTES/s)

I ran a similar test.
Sending system is on synchronous gigabit fibre on US east coast,
receiving system is near London on 110/21 fibre (so, gigabit in the sending
direction):

100% 2000MB   7.2MB/s   04:36

using rsync -azP : 2,097,152,000 100%    6.81MB/s    0:04:53 (xfr#1, to-chk=0/1)

the speed fluctulates a lot. Both systems are quiet in a network and OS sense
for the duration of the test.

>The London box is pretty old, and is a virtual host scheduled to be decomissioned.
>It is running an old openssl 1.X, openssh 8.8 and is a single core 2.4Ghz amd64 box.
>
>The France box is a 4 core bare metal 3.1Ghz and64 running openssh 9.2 and openssl 1.1.1

both ends here are running very recent -current, so ssl/ssh is 
OpenSSH_9.3p1, OpenSSL 3.0.9 30 May 2023

>Anything more I can tell you that may help?

Thanks very much for your input. I'm certain it's not a freebsd problem.

--