Re: Very slow scp performance comparing to Linux
- Reply: Mark Millard : "Re: Very slow scp performance comparing to Linux [dd to /dev/null shows substantial FreeBSD vs. Ubuntu differences for bs=1k (or 1K) and bs=512]"
- In reply to: Mark Millard : "Re: Very slow scp performance comparing to Linux"
- Go to: [ bottom of page ] [ top of archives ] [ this month ]
Date: Wed, 30 Aug 2023 08:49:54 UTC
On Aug 30, 2023, at 01:22, Mark Millard <marklmi@yahoo.com> wrote: > On Aug 30, 2023, at 01:17, Mark Millard <marklmi@yahoo.com> wrote: > >> On Aug 29, 2023, at 12:52, Mark Millard <marklmi@yahoo.com> wrote: >> >>> Wei Hu <weh_at_microsoft.com> wrote on >>> Date: Tue, 29 Aug 2023 12:55:35 UTC : >>> >>>> Thanks for the update. Seems the numbers are the same on zfs and ufs. That's >>>> good to know. >>>> >>>> Yes, your numbers on ARM64 are better than mine on Intel. However, my original >>>> intention was to find out why scp on Linux is performing much better than FreeBSD >>>> under the same hardware env. >>>> >>>> Is it possible to try Linux in your ARM64 setting? I am using Ubuntu 22.04 on ext4 >>>> file system. >>> >>> >>> I tried to use the Hyper-V Quick Create on the Windows Dev Kit 2023 >>> to install a Ubuntu 22.04 . (No clue if ext4 would result.) But the >>> Hyper-V UEFI reports for the disk created: >>> >>> 1. SCSI Disk 0,0 >>> The boot loader did not load an operating system. >>> >>> (It then reports the network adapter attempt found no >>> boot image, but that is expected.) >>> >>> That leaves me wondering if Hyper-V Quick Create >>> established a VM file holding Intel/AMD material >>> despite the aarch64 context. >>> >>> Establishing a Ubuntu more directly is not familiar and >>> will have to be a background activity and, so, likely >>> will not be timely. If I did any experiments outside >>> Hyper-V (native booting), they would be with slower >>> USB3 SSD media than I use for FreeBSD. >>> >>> I did notice that Hyper-V Quick Create did not create >>> a fixed sized disk but a dynamic sized one. That is >>> different than what I did for FreeBSD. >>> >>> Also, it was not obvious if you were after aarch64 >>> Hyper-V testing vs. native-boot testing vs. both. So >>> I may have gone the wrong direction from the start. >>> It is possible that I'd find establishing a native-boot >>> easier and then be able to have a VM file created from >>> the media, more like what I did with FreeBSD. >>> >>> The Ubuntu activity likely would not be analogous to >>> the FreeBSD builds having -mcpu= optimization used. >>> >>> Back to $work. >>> >> >> I found a sequence of UI operations that worked for >> installing Ubuntu server 22.04.3 into Hyper-V in >> Windows 11 Pro on the Windows Dev Kit 2023 via >> use of a downloaded *.iso . >> >> The kernel that results predates 6.0: >> >> $ uname -ap >> Linux ubwdk23s 5.15.0-82-generic #91-Ubuntu SMP Mon Aug 14 14:19:18 UTC 2023 aarch64 aarch64 aarch64 GNU/Linux >> >> Using my usual rule of rebooting before the first scp: >> >> $ scp FreeBSD-14.0-ALPHA2-arm-armv7-GENERICSD-20230818-77013f29d048-264841.img markmi@localhost:FreeBSD-14-TEST.img >> . . . >> FreeBSD-14.0-ALPHA2-arm-armv7-GENERICSD-20230818-77013f29d048-264841.img 100% 5120MB 431.3MB/s 00:11 >> >> $ rm FreeBSD-14-TEST.img >> $ scp FreeBSD-14.0-ALPHA2-arm-armv7-GENERICSD-20230818-77013f29d048-264841.img markmi@localhost:FreeBSD-14-TEST.img >> . . . >> FreeBSD-14.0-ALPHA2-arm-armv7-GENERICSD-20230818-77013f29d048-264841.img 100% 5120MB 482.2MB/s 00:10 >> >> Definitely faster than the FreeBSD results that I reported >> earlier, including faster than the ThreadRipper 1950X with >> Optane in a PCIe slot (more like 300 MiBytes/sec). >> >> I again used 6 cores, 24576 MiBytes of RAM, a fixed sized virtual hard >> disk under Hyper-V. >> >> For reference: >> >> $ lsblk -f >> NAME FSTYPE FSVER LABEL UUID FSAVAIL FSUSE% MOUNTPOINTS >> loop0 squashfs 4.0 0 100% /snap/core20/1977 >> loop1 squashfs 4.0 0 100% /snap/lxd/24326 >> loop2 squashfs 4.0 0 100% /snap/snapd/19459 >> sda ├─sda1 vfat FAT32 F7E9-1344 1G 1% /boot/efi >> └─sda2 ext4 1.0 48a0dbe6-5a99-4b6e-92dc-fe6d8efc6ffe 99.3G 14% / >> >> >> >> An experiment would be to have a small amount if RAM relative >> the file size. That would force it to actually write to media >> for some part of the file copy. > > The wording was poor: "force it" here is just from the > Ubuntu viewpoint. I make no claim to know if Hyper-V > is actually writing the material out to media at the > time vs. later. > >> So using 1024 MiByte of RAM assigned in Hyper-V: >> >> $ scp FreeBSD-14.0-ALPHA2-arm-armv7-GENERICSD-20230818-77013f29d048-264841.img markmi@localhost:FreeBSD-14-TEST.img >> . . . >> FreeBSD-14.0-ALPHA2-arm-armv7-GENERICSD-20230818-77013f29d048-264841.img 100% 5120MB 407.5MB/s 00:12 >> >> $ rm FreeBSD-14-TEST.img >> $ scp FreeBSD-14.0-ALPHA2-arm-armv7-GENERICSD-20230818-77013f29d048-264841.img markmi@localhost:FreeBSD-14-TEST.img >> . . . >> FreeBSD-14.0-ALPHA2-arm-armv7-GENERICSD-20230818-77013f29d048-264841.img 100% 5120MB 404.7MB/s 00:12 >> >> Still definitely faster than the FreeBSD results that I >> reported earlier, including faster than the ThreadRipper >> 1950X with Optane in a PCIe slot (more like 300 MiBytes/sec). One more variation in ubuntu under Hyper-V, still with 1024 MiBytes of assigned RAM: use of localhost:/dev/null $ scp FreeBSD-14.0-ALPHA2-arm-armv7-GENERICSD-20230818-77013f29d048-264841.img markmi@localhost:/dev/null . . . FreeBSD-14.0-ALPHA2-arm-armv7-GENERICSD-20230818-77013f29d048-264841.img $ scp FreeBSD-14.0-ALPHA2-arm-armv7-GENERICSD-20230818-77013f29d048-264841.img markmi@localhost:/dev/null . . . FreeBSD-14.0-ALPHA2-arm-armv7-GENERICSD-20230818-77013f29d048-264841.img 100% 5120MB 492.9MB/s 00:10 The matching FreeBSD examples with 24576 MiBytes of RAM assigned (ZFS context): # scp FreeBSD-14.0-ALPHA2-arm-armv7-GENERICSD-20230818-77013f29d048-264841.img root@localhost:/dev/null . . . FreeBSD-14.0-ALPHA2-arm-armv7-GENERICSD-20230818-77013f29d048-264841.img # scp FreeBSD-14.0-ALPHA2-arm-armv7-GENERICSD-20230818-77013f29d048-264841.img root@localhost:/dev/null . . . FreeBSD-14.0-ALPHA2-arm-armv7-GENERICSD-20230818-77013f29d048-264841.img 100% 5120MB 198.7MB/s 00:25 Note: At most one VM running at a time, never both in overlapping times. === Mark Millard marklmi at yahoo.com