From nobody Tue Aug 29 12:55:35 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RZnTt1Lxlz4rmQn for ; Tue, 29 Aug 2023 12:55:46 +0000 (UTC) (envelope-from weh@microsoft.com) Received: from HK2P15301CU002.outbound.protection.outlook.com (mail-eastasiaazlp170100001.outbound.protection.outlook.com [IPv6:2a01:111:f403:c400::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "DigiCert Cloud Services CA-1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RZnTs25k9z3G5y for ; Tue, 29 Aug 2023 12:55:45 +0000 (UTC) (envelope-from weh@microsoft.com) Authentication-Results: mx1.freebsd.org; none ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=eQXKaxLhT+yQYDSdhDZOYmUI3EI8G29jycdi77ICCPgxI7XGgh2dXOhuSRb0Zd0+wWa4G+1yzTte4C9sSSkGDHnNZoF9fPItFQrlVSwgb1MPcD4T3amVqX/A7bdPGaunSWywSJS4RziM8ZV2fmuoFT49kwQMIQrVczkzYfgFT38tcRiN6TBoHk6HKtftiUGXYAXnAMFN9hDwZYbMxnfuYIUZ5Z+LTjKVR5tiQO9Lm7hsmN3/lizNxvlaTkrMHKvxpZYQwnQBOCavXdqTs70KZBQEppDdI3z6IR8VQ4NI3JiASDqWI4IMZYRJi/mR2mnN7FxK9SQOgwAVOIpbgXlHtA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=AM91Rsn0QFQGPj5eBcbOIC6e7ASPYfGZwgT+PwFDW5Y=; b=CSDJB8S4dH/PIok5WFwhYX9ld5MiOBLmm1BXeR+Cl+sil3e4uK4LJY7Ch+cLZbubh/Ob4W7EKB8nOeNxQKAU/crwh6wN69zaZex81K/wB171ubgpnTOMf0CGtwR09EjCTU6xLsVmFJGZ9v1oDaHcUj4IuWdFi0G3g9285yoNmZY2HihoJR4aI+1gkEPcBnLsIZHG3/mqDaCI1R+dNx0/Xk7Btzk7RqAlGiCIYFXt/TBbrhr4FMoCMXZDh1ihBv2RPgBN4O9nqTlk1TTEsn2ZnyMeV0A5LHdRiVBjmd2qNRVt5SgUKXUZfSaKvhQTm1TXPAE54gknEtZ90ivN65ZlAA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=microsoft.com; dmarc=pass action=none header.from=microsoft.com; dkim=pass header.d=microsoft.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=AM91Rsn0QFQGPj5eBcbOIC6e7ASPYfGZwgT+PwFDW5Y=; b=FOwzFnN/szqL8dAIAV1muJvWc0Bd5FyXNGtiyq6TTS2G2buRyB7+daN+CZMSBzWe32J+254E+kxi4hCheV9YnIDg1BxE6H7lKd2jBinTCtgebwOTec+hi6IwcmEnD6lIm9h8Jcmus1dcaZBnuf7p04raJvTKQoJuM7Or87JXNtA= Received: from SI2P153MB0441.APCP153.PROD.OUTLOOK.COM (2603:1096:4:fc::7) by PSAP153MB0502.APCP153.PROD.OUTLOOK.COM (2603:1096:301:90::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6768.9; Tue, 29 Aug 2023 12:55:37 +0000 Received: from SI2P153MB0441.APCP153.PROD.OUTLOOK.COM ([fe80::7bae:a915:919b:6f83]) by SI2P153MB0441.APCP153.PROD.OUTLOOK.COM ([fe80::7bae:a915:919b:6f83%7]) with mapi id 15.20.6768.007; Tue, 29 Aug 2023 12:55:36 +0000 From: Wei Hu To: Mark Millard CC: FreeBSD Hackers Subject: RE: Very slow scp performance comparing to Linux Thread-Topic: Very slow scp performance comparing to Linux Thread-Index: AQHZ2mNniu2mI7eQ9kazPr8glcZkyrABIWYAgAAYXWA= Date: Tue, 29 Aug 2023 12:55:35 +0000 Message-ID: References: <98C8E07C-2247-4439-8836-ED350CC83F16@yahoo.com> <6E433834-E192-44F1-9FF1-3814F13449FF@yahoo.com> In-Reply-To: <6E433834-E192-44F1-9FF1-3814F13449FF@yahoo.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: msip_labels: MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_ActionId=553017a6-a2be-4a47-b25c-7ef67c14489b;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_ContentBits=0;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Enabled=true;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Method=Standard;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Name=Internal;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SetDate=2023-08-29T12:49:38Z;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SiteId=72f988bf-86f1-41af-91ab-2d7cd011db47; x-ms-publictraffictype: Email x-ms-traffictypediagnostic: SI2P153MB0441:EE_|PSAP153MB0502:EE_ x-ms-office365-filtering-correlation-id: e1b6f4a5-76d7-4d8a-8468-08dba88f3ce0 x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: 7w/RjSqaCN00rd5WqoBo5RQRfG8RCg05ZF+VIv8icyfg/VDxDgWZZQM9y50TrUalTRK6T4JLICkuve0MeQDvpBbqNVB2UNLBahoZZBam+YgOCdLtEECq+fZYx8/S22vo/pD4DouNCbOmEODORVS8VzwKAs67OQc5EFKYSf5IgoUurlqqP1ttSzTFnRdWQI79i1Bn//k/hd3Pgrzz8zCgj5/YbBcKT0p6iKN3xRdqvFwD/n6LMEWOeBoJspDrt4g7aIORHbhyQzkDzzdN8HgSZr6+FL0ZJpxQNL1xlOIyHA4ruOT13Ii6+QXwRu0qLhpBA86ta8Jom6K0U1kxDBtk22cxFw2P5XaUx0rVWvBY7u36lnFG/ZMJQKkyAyWZLnJqEgRqodt5KwybniZkOU7UUryQCMJOA/ygDYaM8awin4aSINin88W8FLrEdz1eIZpf4mgo/OTXSwNzbfnWcsdz4ZFkITjDJBoMzWzRGgTuyCRNRfUiWdO+PgwvbP89dvHokjlmQnmQbywt9ZKktew2TT1TA8gyUf69n3YfDleW18siOfu+XdeA3UAnwpTtwHwZbusaEXuHHRbGF2MHdEjgLgm0FkN88dv7j9tnfhTqVIxiyDmykBxXkWWnZvxFaPWs3XtwZ6zp6knctY3MMVIwqO3hkkwjRqxem+GughwD8gfWp89cdrlgN4uVFS20LG4kFpJwKs65Y5XGqFit67en/Q== x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:SI2P153MB0441.APCP153.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230031)(396003)(366004)(376002)(136003)(39860400002)(346002)(1800799009)(451199024)(186009)(478600001)(86362001)(55016003)(33656002)(2906002)(19627235002)(83380400001)(82960400001)(38070700005)(38100700002)(6916009)(82950400001)(41300700001)(316002)(122000001)(5660300002)(76116006)(52536014)(8676002)(4326008)(8936002)(66946007)(64756008)(66446008)(66476007)(66556008)(53546011)(7696005)(6506007)(12101799020)(71200400001)(8990500004)(9686003)(10290500003);DIR:OUT;SFP:1102; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?PCA7r1bxG56jJJtvFO4VhqUiXPlWMT+gYOe8RwHeOHNFa8ekf/GLwPQ5t/2c?= =?us-ascii?Q?kV+g1uCFVj71g+D9/8QlAZ9MfimkOkupNP+CUhwI5C7oBHgXPZbVtCVmSRLb?= =?us-ascii?Q?dacbOIT5NsjBgdwWKtUUtRuvYNVamawkZMYOle1IXm5F8sJH2y8xQqlYl96v?= =?us-ascii?Q?h67+AvOCJQQ8IewsQEB5nlSpx1DAuuVz9JLWspGGRZfKD7rZi6dldVzCIZne?= =?us-ascii?Q?HLeHoEZKBaGpDL4Au1L/nEmES6Ae1HWVdoXK2YLLJVDo/vxr4hl1Unw86NTC?= =?us-ascii?Q?hS5R4D1tCtz8h6vIG4iaSTTmzx5Jfibx1MiOKS1u4Lj9ZFF31q+Io+XjX6h1?= =?us-ascii?Q?XYJkgPOjQeusduzZprmd2ujCT2+cjMZJ+Akf1pE+5iQpOO6r+FW3302ktcyF?= =?us-ascii?Q?RMToMBOtHXUckxKVyHYeF4XUJGG0el8egHOwwU7b5WsaJarVwelYaUYq2Pre?= =?us-ascii?Q?JbB4VEj6WFJDMyHM35LGMvmInNnH2Ux0uS19NvSa9Vj+W98KJk7VqwvFjJAW?= =?us-ascii?Q?PA/HpsM/JlRK+4qE0x57Fs5gNieSGscjuM2PzUqscx0R5V4Sq2hEYGlol47x?= =?us-ascii?Q?Yn2Xv+dY9eXmvg5cLjB5DiAEpawEykR2XVFlxS03xrOOnks+CcCPdg3o1bz4?= =?us-ascii?Q?OS37uDPI9TRA9cEd9UvWJdc0uadP3SVEIbvRDP1T+GZJmGHfRNCZCU6LHWhr?= =?us-ascii?Q?EaJPICUlI/CfznyggjlbWCL0baiV5YmF3sqBVvajsVHEjioJSF1zFQ52/dUz?= =?us-ascii?Q?DJra3nnqvhuo5SY6iYNm2hNpH6g4iazLQRZyNCth2AF2LyuUeM+t9nfVbDsj?= =?us-ascii?Q?fj/z4m7pfKq9iTuucMITk86iBtcS+nIBlg8RqqKA+QxwtHcHv6lRmIiUBIND?= =?us-ascii?Q?/p9CmTUPVSwXbJyE5lDZntIV79DsUAAXnvlK42TCrZAQ1dkjoSt5mo6zWOaQ?= =?us-ascii?Q?z3M/CBxfIOaxpvbgxCfI36+sqlcNOfpV+eR0ovaMS35D5/9VWKjBU+YhajTz?= =?us-ascii?Q?a/bArUkvyo2jeqviWaeeB4yrira2ttLQ+yEaAdvNriVQekPYbeosVpBrHECO?= =?us-ascii?Q?4AyiSWbATGMvAyshlADdnffStk2ZRSWYuQ5llskMXrWI17/OyavXPS+a/7+4?= =?us-ascii?Q?VmGFE7pa3ucFf8pDKHzOzfcQHmdIJmpyD9SICaQwS2Ov5dqDi8FWDr5NeJtC?= =?us-ascii?Q?JyQRkMd4eXsB1dPmmuEOSbGaK+Em8Qu/TO8hh5M8ovZklEPvXMNpNqaEMEwn?= =?us-ascii?Q?Q8o1q1ULaHWvmWy+Anpt8yXIQANMqHcFzBJBFkhkrLSWgd7/z7aM8OLKEP7g?= =?us-ascii?Q?JgGiXCdTEpMQSv+4HE43yP3djSo0BlFIG9rN3iEqGQBcqq4fl4Bw09zrpbMB?= =?us-ascii?Q?FK2/TRRVKRKoDCiRVj1L6JgZMbNv3HzokyL5/htMxVpsOaoTPvZBnt5NNIxu?= =?us-ascii?Q?P6s9L6Sl4L0goN4hD+PE8t2s5mRjlYeBX5Gq5fNeRQNZat+SbzPnUg0hM+5B?= =?us-ascii?Q?HJkCoZNdExt/+LmaAC2Co89SLSvElC34SqHiHB5NKtF8Nan02pUOPP3XbrVU?= =?us-ascii?Q?XnJmFL78igtcgxF2o5ENgtX23At7COkF5LwQrQN/lzZUra7z1/IZ5VuO8zl8?= =?us-ascii?Q?aw=3D=3D?= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 X-OriginatorOrg: microsoft.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: SI2P153MB0441.APCP153.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-Network-Message-Id: e1b6f4a5-76d7-4d8a-8468-08dba88f3ce0 X-MS-Exchange-CrossTenant-originalarrivaltime: 29 Aug 2023 12:55:35.8077 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 72f988bf-86f1-41af-91ab-2d7cd011db47 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: TjrFpA45XmN0mUGVxRIN+CxT8DQG45JxhMRcf0XpX6tX6LmXUwFjs5sIXUMbK0CxoVmIIFeJDwVXd5Bn0qQPRw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PSAP153MB0502 X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:8075, ipnet:2a01:111:f000::/36, country:US] X-Rspamd-Queue-Id: 4RZnTs25k9z3G5y Hi Mark, Thanks for the update. Seems the numbers are the same on zfs and ufs. That'= s=20 good to know.=20 Yes, your numbers on ARM64 are better than mine on Intel. However, my origi= nal intention was to find out why scp on Linux is performing much better than F= reeBSD under the same hardware env.=20 Is it possible to try Linux in your ARM64 setting? I am using Ubuntu 22.04 = on ext4=20 file system. Thanks, Wei=20 > -----Original Message----- > From: Mark Millard > Sent: Tuesday, August 29, 2023 7:22 PM > To: Wei Hu > Cc: FreeBSD Hackers > Subject: Re: Very slow scp performance comparing to Linux >=20 > [Adding USB3/U.2 Optane UFS Windows Dev Kit 2023 scp examples, no VM's > involved.] >=20 > On Aug 29, 2023, at 03:27, Mark Millard wrote: >=20 > > Wei Hu wrote on > > Date: Tue, 29 Aug 2023 07:07:39 UTC : > > > >> Sorry for the top posting. But I don't want to make it look too > >> messy. Here is the Information that I have missed in my original email= . > >> > >> All VMs are running on Intel(R) Xeon(R) Platinum 8473C (2100.00-MHz > K8-class CPU). > >> > >> FreeBSD VMs are 16 vcpu with 128 GB memory, in non-debug build: > >> 14.0-ALPHA1 FreeBSD 14.0-ALPHA1 amd64 1400094 #7 > >> nodbg-n264692-59e706ffee52-dirty... > >> /usr/obj/usr/src/main/amd64.amd64/sys/GENERIC-NODEBUG amd64 > >> > >> Ubuntu VMs are 4 vcpu with 32 GB memory, kernel version: > >> 6.2.0-1009-azure #9~22.04.3-Ubuntu SMP Tue Aug 1 20:51:07 UTC 2023 > >> x86_64 x86_64 x86_64 GNU/Linux > >> > >> I did a couple more tests as suggested by others in this thread. In re= cap: > >> > >> Scp to localhost, FreeBSD (ufs) vs Ubuntu (ext4): 70 MB/s vs 550 MB/s > >> Scp to localhost, FreeBSD (tmpfs) vs Ubuntu (tmpfs): 630 MB/s vs 660 > >> MB/s > >> > >> Iperf3 single stream to localhost: FreeBSD vs Ubuntu: 30.9 Gb/s vs > >> 48.8 Gb/s > >> > >> Would these numbers suggest that > >> 1. ext4 caches a lot more than ufs? > >> 2. there is a tcp performance gap in the network stack between FreeBSD > and Ubuntu? > >> > >> Would you also try run scp on ufs on your bare metal arm host? I am > curious to now how different between ufs and zfs. > > > > > > For this round I'm rebooting between the unxz and the 1st scp. > > So I'll also have zfs results again. I'll also do a 2nd scp (no > > reboot) to see if it gets notably different results. > > > > . . . > > > > Well, I just got FreeBSD main [so: 15] running under HyperV on the > > Windows Dev Kit 2023. So reporting for there first. This was via an > > ssh session. The context is ZFS. The VM file size is fixed, as is the > > RAM size. > > 6 cores (of 8) and 24576 MiBytes assigned (of 32 > > GiBytes) to the one FreeBSD instance. The VM file is on the internal > > NVMe drive in the Windows 11 Pro file system in the default place. > > > > (I was having it copy the hardrive media to the VM file when I started > > this process. Modern HyperV no longer seems to support direct use of > > USB3 physical media. I first had to produce a copy of the material on > > smaller media so that a fixed VM file size from a copy to create the > > VM file would fit in the NVMe's free space.) > > > > # uname -apKU > > FreeBSD CA78C-WDK23s-ZFS 15.0-CURRENT FreeBSD 15.0-CURRENT > aarch64 1500000 #13 main-n265027-2f06449d6429-dirty: Fri Aug 25 > 09:20:31 PDT 2023 root@CA78C-WDK23-ZFS:/usr/obj/BUILDs/main- > CA78C-nodbg-clang/usr/main-src/arm64.aarch64/sys/GENERIC-NODBG- > CA78C arm64 aarch64 1500000 1500000 > > > > (The ZFS content is a copy of the USB3 interfaced ZFS Optane media's > > content previously reported on. > > So the installed system was built with -mcpu=3D based optimization, as > > noted before.) > > > > # scp > > FreeBSD-14.0-ALPHA2-arm-armv7-GENERICSD-20230818-77013f29d048- > 264841.i > > mg root@localhost:FreeBSD-14-TEST.img > > . . . > > FreeBSD-14.0-ALPHA2-arm-armv7-GENERICSD-20230818-77013f29d048- > 264841.img = 100% 5120MB > 193.6MB/s 00:26 > > > > # rm ~/FreeBSD-14-TEST.img > > # scp > > FreeBSD-14.0-ALPHA2-arm-armv7-GENERICSD-20230818-77013f29d048- > 264841.i > > mg root@localhost:FreeBSD-14-TEST.img > > . . . > > FreeBSD-14.0-ALPHA2-arm-armv7-GENERICSD-20230818-77013f29d048- > 264841.img = 100% 5120MB > 198.0MB/s 00:25 > > > > > > So, faster than what you are reporting for the > > Intel(R) Xeon(R) Platinum 8473C (2100.00-MHz K8-class CPU) context. > > > > For reference: > > > > # gpart show -pl > > =3D> 40 468862055 da0 GPT (224G) > > 40 32728 - free - (16M) > > 32768 102400 da0p1 wdk23sCA78Cefi (50M) > > 135168 421703680 da0p2 wdk23sCA78Czfs (201G) > > 421838848 47022080 da0p3 wdk23sCA78Cswp22 (22G) > > 468860928 1167 - free - (584K) > > > > # zpool list > > NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP > HEALTH ALTROOT > > zwdk23s 200G 79.8G 120G - - 0% 39% 1.00x = ONLINE - > > > > (UFS would have notably more allocated and less free for the same size > > partition.) > > > > > > > > The below is be based on the HoneyComb (16 cortex-a72's) since I've > > got the HyperV context going on the Windows Dev Kit 2023 at the > > moment. > > > > > > UFS first: > > > > # uname -apKU > > FreeBSD HC-CA72-UFS 15.0-CURRENT FreeBSD 15.0-CURRENT aarch64 > 1500000 #110 main-n265027-2f06449d6429-dirty: Fri Aug 25 09:19:53 PDT > 2023 root@CA72-16Gp-ZFS:/usr/obj/BUILDs/main-CA72-nodbg- > clang/usr/main-src/arm64.aarch64/sys/GENERIC-NODBG-CA72 arm64 > aarch64 1500000 1500000 > > > > # scp > > FreeBSD-14.0-ALPHA2-arm-armv7-GENERICSD-20230818-77013f29d048- > 264841.i > > mg root@localhost:FreeBSD-14-TEST.img > > . . . > > FreeBSD-14.0-ALPHA2-arm-armv7-GENERICSD-20230818-77013f29d048- > 264841.img = 100% 5120MB > 129.7MB/s 00:39 > > > > # rm ~/FreeBSD-14-TEST.img > > # scp > > FreeBSD-14.0-ALPHA2-arm-armv7-GENERICSD-20230818-77013f29d048- > 264841.i > > mg root@localhost:FreeBSD-14-TEST.img > > . . . > > FreeBSD-14.0-ALPHA2-arm-armv7-GENERICSD-20230818-77013f29d048- > 264841.img = 100% 5120MB > 130.9MB/s 00:39 > > > > > > So, faster than what you are reporting for the > > Intel(R) Xeon(R) Platinum 8473C (2100.00-MHz K8-class CPU) context. > > > > Note: This is via a U.2 Optane 960 GB media and an M.2 adapter instead > > of being via a PCIe Optane 960 GB media in the PCIe slot. > > > > > > ZFS second: > > > > # uname -apKU > > FreeBSD CA72-16Gp-ZFS 15.0-CURRENT FreeBSD 15.0-CURRENT aarch64 > 1500000 #110 main-n265027-2f06449d6429-dirty: Fri Aug 25 09:19:53 PDT > 2023 root@CA72-16Gp-ZFS:/usr/obj/BUILDs/main-CA72-nodbg- > clang/usr/main-src/arm64.aarch64/sys/GENERIC-NODBG-CA72 arm64 > aarch64 1500000 1500000 > > > > # scp > > FreeBSD-14.0-ALPHA2-arm-armv7-GENERICSD-20230818-77013f29d048- > 264841.i > > mg root@localhost:FreeBSD-14-TEST.img > > . . . > > FreeBSD-14.0-ALPHA2-arm-armv7-GENERICSD-20230818-77013f29d048- > 264841.img = 100% 5120MB > 121.1MB/s 00:42 > > > > # rm ~/FreeBSD-14-TEST.img > > # scp > > FreeBSD-14.0-ALPHA2-arm-armv7-GENERICSD-20230818-77013f29d048- > 264841.i > > mg root@localhost:FreeBSD-14-TEST.img > > (root@localhost) Password for root@CA72-16Gp-ZFS: > > FreeBSD-14.0-ALPHA2-arm-armv7-GENERICSD-20230818-77013f29d048- > 264841.img = 100% 5120MB > 124.6MB/s 00:41 > > > > > > So, faster than what you are reporting for the > > Intel(R) Xeon(R) Platinum 8473C (2100.00-MHz K8-class CPU) context. > > > > Note: This is via a PCIe Optane 960 GB media in the PCIe slot. > > > > > > UFS was slightly faster then ZFS for the HoneyComb context but there > > is the M.2 vs. PCIe difference as well. > > >=20 > # uname -apKU > FreeBSD CA78C-WDK23-UFS 15.0-CURRENT FreeBSD 15.0-CURRENT aarch64 > 1500000 #13 main-n265027-2f06449d6429-dirty: Fri Aug 25 09:20:31 PDT > 2023 root@CA78C-WDK23-ZFS:/usr/obj/BUILDs/main-CA78C-nodbg- > clang/usr/main-src/arm64.aarch64/sys/GENERIC-NODBG-CA78C arm64 > aarch64 1500000 1500000 >=20 > Again, a -mcpu=3D optimized build context for the FreeBSD in > operation. >=20 > (Still rebooting first. Then . . .) >=20 > # scp FreeBSD-14.0-ALPHA2-arm-armv7-GENERICSD-20230818- > 77013f29d048-264841.img root@localhost:FreeBSD-14-TEST.img > . . . > FreeBSD-14.0-ALPHA2-arm-armv7-GENERICSD-20230818-77013f29d048- > 264841.img = 100% 5120MB > 199.3MB/s 00:25 >=20 > # rm ~/FreeBSD-14-TEST.img > # scp FreeBSD-14.0-ALPHA2-arm-armv7-GENERICSD-20230818- > 77013f29d048-264841.img root@localhost:FreeBSD-14-TEST.img > . . . > FreeBSD-14.0-ALPHA2-arm-armv7-GENERICSD-20230818-77013f29d048- > 264841.img = 100% 5120MB > 204.9MB/s 00:24 >=20 >=20 > So, faster than what you are reporting for the > Intel(R) Xeon(R) Platinum 8473C (2100.00-MHz K8-class CPU) > context. >=20 > The Windows Dev Kit 2023 figures are generally faster than the > HoneyComb figures. >=20 > =3D=3D=3D > Mark Millard > marklmi at yahoo.com