Re: Very slow scp performance comparing to Linux

Hi,

What is the drive back-end of your machines? I mean, what is seem to be 
fast copied could still be cached in filesystem/storage buffers but not 
actually written on the disk.

Best,
Mike


On 8/28/2023 9:32 AM, Wei Hu wrote:
> Hi,
>
> When I was testing a new NIC, I found the single stream scp performance was almost 8 time slower than Linux on the RX side. Initially I thought it might be something with the NIC. But when I switched to sending the file on localhost, the numbers stay the same.
>
> Here I was sending a 2GB file from sender to receiver using scp.  FreeBSD is a recent NON-DEBUG build from CURRENT. The Ubuntu Linux kernel is 6.2.0. Both run in HyperV VMs on the same type of hardware. The FreeBSD VM has 16 vcpus, while Ubuntu VM has 4 vcpu.
>
> Sender		Receiver	throughput
> Linux		FreeBSD 	70 MB/s
> Linux		Linux 		550 MB/s
> FreeBSD	FreeBSD	70 MB/s
> FreeBSD	Linux		350 MB/s
> FreeBSD  	localhost	70 MB/s
> Linux		localhost	550 MB/s
>
>  From theses test, it seems I can rule out the issue on NIC and its driver. Looks the FreeBSD kernel network stack is much slower than Linux on single stream TCP, or there are some problem with scp?
>
> I also tried turning on following kernel parameters on FreeBSD kernel. But it makes no difference, neither do the other tcp cc algorithms such as htcp and newreno.
>
> net.inet.tcp.soreceive_stream="1"
> net.isr.maxthreads="-1"
> net.isr.bindthreads="1"
>
> net.inet.ip.intr_queue_maxlen=2048
> net.inet.tcp.recvbuf_max=16777216
> net.inet.tcp.recvspace=419430
> net.inet.tcp.sendbuf_max=16777216
> net.inet.tcp.sendspace=209715
> kern.ipc.maxsockbuf=16777216
>
> Any ideas?
>
> Thanks,
> Wei
>