From nobody Sat Sep 03 03:41:06 2022 X-Original-To: freebsd-fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4MKLD6094Fz4cCq1 for ; Sat, 3 Sep 2022 03:41:10 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from CAN01-YQB-obe.outbound.protection.outlook.com (mail-yqbcan01on2071.outbound.protection.outlook.com [40.107.116.71]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "DigiCert Cloud Services CA-1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4MKLD45N0gz3XR8; Sat, 3 Sep 2022 03:41:08 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Xfv9kOBNjx8l8hgmUqalee+mEMJ21tXKmLVBqfvqbSqXEMVeYGT+Ez4spq+g+KLpAgJE//D1bVYWpTFID9KhVePmdqh5lIDwfpZ9PJDujuUz6AacCn2OJbW0nlpMXNzrFvWVBJkX5LIdiRfq1N57rsj+CqF0YuGWQx3OGw+eXbhiBtySK6I8C2txMZGlyG1Cnu++zNRpQCeQQbCg8cyX5BMGJPg7neRMk/NP0MldDGGb8i96DmUVWG8F018zqKmGgph+4mv3uvvbrzCbtC4MCokP1CaYA6YyCYBxRVwcFE3gm7uSPLKk4GzU6/RT6WyrTOA6SPhnEHHVNs183Ex/bA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Q0AOUSn698LkW4aHhOfKaNSPO9SmlbXuzHKVodbLlas=; b=LiMAL+hheqhxbi4cCXpCvolQJegux2VSUGXq+GyxqyN0dopp87qtzZI0ZE1H2B+LmIsTo6Rgmth6X4RsL7TUwiXqm/j/MPx5kQ93O7rf9Sp5k9+ccvRi7QiFxQbFY4iBx3GZdxJO8BtZqGRsMM3h4adrFMd5KUIoYFR70jhX7ND0ykBMExLE5396ttOoVEfu7Z6xoNP27nwwNd7/TmOUhckviFlUWlln/qhyu3AEhU1fX1fxFeGY7Bg6j7AWryAZL6lYcTZ7Zzry+BHLgIoJNJ8EHUoKQRGRf4CgYfYWzv0OsvymxDcHVTXgjwTtVgi0qltxj6c35kijCOiHNJ92fA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=uoguelph.ca; dmarc=pass action=none header.from=uoguelph.ca; dkim=pass header.d=uoguelph.ca; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=uoguelph.ca; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Q0AOUSn698LkW4aHhOfKaNSPO9SmlbXuzHKVodbLlas=; b=lHWOzwx3BKZLq5wqstwbLGGCkzee8+dLXWzRIN+69q7yBRM7GSmeY+T05c88JWoy8Q65p29Kn6F19mgsCN7TbAmwtKFW+irGdgTU/bLadMcjPfywBBBDn93yHVrHRcWv62DSwVrHhULAKc5Xmj+ZtWRQbhcKJGp1kN4vFjb1ZwNro643LZ5YLWEo3eLMyJlZjXnTfjJpDFaqJ5I/X3R8w9Rc/YF2zNAEbOwluWNXuefS/LHekICT+0HgTTmIxI95h1uKVYPm2S0Ce1qlEZXe+6cREj7IYDWHHyrBY4/QwLJxaRxS+I6fgNHs+UJRm5c9mmKH7daNMXKWUr5LchNdkw== Received: from YQXPR01MB4150.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:c01:6::7) by YT3PR01MB6405.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:70::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5588.10; Sat, 3 Sep 2022 03:41:06 +0000 Received: from YQXPR01MB4150.CANPRD01.PROD.OUTLOOK.COM ([fe80::980d:23ea:9bc2:9f37]) by YQXPR01MB4150.CANPRD01.PROD.OUTLOOK.COM ([fe80::980d:23ea:9bc2:9f37%9]) with mapi id 15.20.5588.016; Sat, 3 Sep 2022 03:41:06 +0000 From: Rick Macklem To: Alan Somers CC: FreeBSD Filesystems , Alexander Motin , Mateusz Guzik Subject: Re: RFC: multiple concurrent I/O ops for copy_file_range(2) Thread-Topic: RFC: multiple concurrent I/O ops for copy_file_range(2) Thread-Index: AQHYv0H/+RMlpemwREiRz9tUi24+Ua3NCN0AgAAEBfo= Date: Sat, 3 Sep 2022 03:41:06 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: msip_labels: x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 5df08924-55da-4028-98e4-08da8d5e2247 x-ms-traffictypediagnostic: YT3PR01MB6405:EE_ x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: Eb6ecSDlWOGQWFAx7wZja4/ginH1M/L9YDNqWQOrTNVIxcoKdFb2GLpGxMTgb78HSEIm5cqYFeUxjJOPUHKIsXSdkvmqla9VUrEKEfdfFX5m+SgKgmCb64PVP3MhKL8FvYNNoIeeyQZsQIRJNkk/5H7APQnfuMoWF8eKNSppE1tuCbw2G2mY72RpUeIr5Don4916Y5ufhfJMN/nf4hDsGwP/TCVgUVASCVqrgxcqQ9vBZFq5PFZGKnbgbxo4GlUoUgP4Z9y/ytjBGKDGoDbXbGUtV11b/A3Mdwva0z8XieoMeikv+LYVFS3mTsoy1FXksvbmqKH8Wduw6HJdzE48kQJVK/7Yhc2B4nk9VQ/xbIrtmI8BTgfx49WgcB2n88ezV6hdcqYJnXEZCD5v3uXFXAvsjBXxPBUUeqS+kUJrimH8fHEY1H2skA+f53pXrqrmDM0hQSnhHGNKxsKVkXiNlvebSiogGmjNQx5mcF0dzAtjYJP1qPHVvLcDtOdYfezQGnWokLXjvovQAGfbBYsSQsId6EDKU3Y9K1HyenAKS+tniuJWVnrsaD9XpeHTojmDV6CwBJdtQRAtYTenCBuWShs0CPOzbuXClSTnocutpjnovjoqI3Rtk1xOOSNic3mbeKyxAuzgd0GDyMAp6iUhtMrjPtM5TR9ManTlkVzEv5+YZ4/Ai4MuaFOQncuwYVBTc6hQoMTTdzJM5f/WizOtW+46/AOd8XX5LpUoV9KVHoDHnbSbq3g9OLfHaIABunsb0a6PGB4dGcUEAK3qcK6qwwaRGwxydPA/JoMdw8Vyrgw= x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:YQXPR01MB4150.CANPRD01.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230016)(4636009)(346002)(39860400002)(376002)(396003)(136003)(366004)(966005)(76116006)(786003)(8676002)(66446008)(316002)(66476007)(4326008)(64756008)(66556008)(66946007)(122000001)(38070700005)(38100700002)(33656002)(86362001)(83380400001)(71200400001)(7696005)(478600001)(41320700001)(6916009)(54906003)(41300700001)(55016003)(2906002)(52536014)(8936002)(9686003)(6506007)(186003)(91956017)(5660300002);DIR:OUT;SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?iso-8859-1?Q?54iHmskwzw7+NYDYVSUL5zeXMvHjGwBwzXJbPlqANgZOnhiZA2sK1o6oXm?= =?iso-8859-1?Q?yTJ4GJxZBrB8w66rUOzqidT1yf9k1Pe/KZC3uFzzxXpbLNnD9eSQhl9M9Z?= =?iso-8859-1?Q?SjcJ7+Z+P+Pe49I/qJc7IwPVU6B5LUH4HBqgZ7vH2sNsdFLjwO4GUXZuvz?= =?iso-8859-1?Q?+Weoy+PsxrLS+VuQlCLleS/dT24kh6z/dx9A2z73ZAYKZWjyD1B0yb7PPJ?= =?iso-8859-1?Q?gm6BfBGz6Ch0Mt2FpSIKtttReJ93mlgvVs2EMMP68ez3ehfh3eplgYtQA6?= =?iso-8859-1?Q?OiIk5nEOsa/DKtBhgjzut1Lj64ELqUgL7ner3+o6+wurHgnM20VJ1slWVI?= =?iso-8859-1?Q?2X4QnhJ/qaKjLQGhZfurJOA6leQQ0DTvw5c9uNqtx5Eu4l3al94MDfHFe9?= =?iso-8859-1?Q?QDkHdLdO/LxR89HqQujCwaSxC2Y76bhJuDNoiTGhQURf0y4GEWjelFrc9V?= =?iso-8859-1?Q?qjGC3oP0YYL9H3rZZg/Z82xrzsrdycTw8l7VB+QLYnb8PoPoTm4DZW5iMR?= =?iso-8859-1?Q?T2u0uZqOMMlSLTMmjLtqIkad861utye3mTYmtzV4vQkPu5S3cVCghoNAc8?= =?iso-8859-1?Q?H1q/Wqpp1maXppNDlTRh8spON/+dsrbq/GsPeOSd3AdZpbA46C/TX4XT/w?= =?iso-8859-1?Q?8rmRcca7QtpP1fIOgTn+W8d5Mqd5Vw9oGr0bYxeQRhiZdaRkjr4Can7P53?= =?iso-8859-1?Q?lQkLMgu9JnwY7qvsvapYb9DIddCdKLEfkS5UFTN09a6iEZybhdy3Kv0Nvd?= =?iso-8859-1?Q?N9LQosAjOZY47MAFYbEZnJLuV0HCWYC/BT0M1M4yKr78GFbEMfj3XXMNkd?= =?iso-8859-1?Q?oK4m5z4bFK5C6euB8kSpL4spQzBN887Ezmsjr0oBdpFkgraKRXxSvSENL/?= =?iso-8859-1?Q?/VfC0ystdwSrnSdyvuRjt0wAHU7nfWRIUKG9SemhgWMu99MZL3fM6S93vP?= =?iso-8859-1?Q?KiIZCKAFP4CrjeKoAqUt2EtP/gp4C5DJhf9YfP7Je6wWxp2b64BErQSAvb?= =?iso-8859-1?Q?nf1r87LoNCm8SON7UcR3H0CmX4KL/YLXtWZLM8cg/T/N/hc9Gx+Cr5eHkU?= =?iso-8859-1?Q?B5W7JwBGgaZezhPma21+//gyPoAKl3Wqa265aOmusBr4xe/vpze/Yfww39?= =?iso-8859-1?Q?eVxCcbEPYoUfdLepVFw1sqb3F6Ys+SV3dUA7ExrR/IB2kJ65xnawbzK/f9?= =?iso-8859-1?Q?CLVo8Zr5/S3wMhProAr6M7MaQIIfCzCLU7+pcq2udDmCNFIQ0IIl7jgcPc?= =?iso-8859-1?Q?71KSOV+iBt3vBo8Jvj2wSl9QywBANxgFJpucHc/cQX8jF3dbEZJ4WZMGm3?= =?iso-8859-1?Q?Drs2lTDYdGzuCETyUW5gzAo6L/9RWVnXxlRtz4GsNxUYZiT8WYe7g0oGLQ?= =?iso-8859-1?Q?iIv+CH1Jdu+To9fC6P0qG0L2RiocMX0aDAYVdhFicYcQc0zfFIx8D6l1V9?= =?iso-8859-1?Q?qfM6tuf5o9qXqA3qB3V1uPFspjfmGwFiv7eK3IYluSBGRzciKNDHxz6K8g?= =?iso-8859-1?Q?jet8kLrkDxH9YpY5wYlW6Oz9fsQXMnTB5DRIPwXURWuEMlWjq6ylqrAT4x?= =?iso-8859-1?Q?llbipZw8Ax9MRGpKi3jFCZaiMA7dt+Q6xoB+IQm18GSmy2wEDTKUwjGyLO?= =?iso-8859-1?Q?cl9/UhDQyd1W93e13f46VRQMzBbIV4q2gPjIO76MHo/IdSIvIcma/CV+QM?= =?iso-8859-1?Q?YIT4C0IWgK8zR4JPJnJ61ekSreNLMD7R2WU5JJWO?= Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org MIME-Version: 1.0 X-OriginatorOrg: uoguelph.ca X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: YQXPR01MB4150.CANPRD01.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-Network-Message-Id: 5df08924-55da-4028-98e4-08da8d5e2247 X-MS-Exchange-CrossTenant-originalarrivaltime: 03 Sep 2022 03:41:06.7438 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: be62a12b-2cad-49a1-a5fa-85f4f3156a7d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: xWTDxs19EALIXxPGXKeybNUv/caHnN0rNmd7GIh8Fujiiny7eyoUiacnfiu7uwkAtdX3Jx1yX9tKm1C1hejeuw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: YT3PR01MB6405 X-Rspamd-Queue-Id: 4MKLD45N0gz3XR8 X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=uoguelph.ca header.s=selector2 header.b=lHWOzwx3; arc=pass ("microsoft.com:s=arcselector9901:i=1"); dmarc=pass (policy=none) header.from=uoguelph.ca; spf=pass (mx1.freebsd.org: domain of rmacklem@uoguelph.ca designates 40.107.116.71 as permitted sender) smtp.mailfrom=rmacklem@uoguelph.ca X-Spamd-Result: default: False [-6.00 / 15.00]; ARC_ALLOW(-1.00)[microsoft.com:s=arcselector9901:i=1]; DWL_DNSWL_LOW(-1.00)[uoguelph.ca:dkim]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.998]; DMARC_POLICY_ALLOW(-0.50)[uoguelph.ca,none]; R_DKIM_ALLOW(-0.20)[uoguelph.ca:s=selector2]; R_SPF_ALLOW(-0.20)[+ip4:40.107.0.0/16]; MIME_GOOD(-0.10)[text/plain]; FROM_HAS_DN(0.00)[]; MLMMJ_DEST(0.00)[freebsd-fs@freebsd.org]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:8075, ipnet:40.104.0.0/14, country:US]; MIME_TRACE(0.00)[0:+]; RWL_MAILSPIKE_POSSIBLE(0.00)[40.107.116.71:from]; DKIM_TRACE(0.00)[uoguelph.ca:+]; FREEMAIL_CC(0.00)[freebsd.org,FreeBSD.org,gmail.com]; RCPT_COUNT_THREE(0.00)[4]; FREEFALL_USER(0.00)[rmacklem]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[] X-ThisMailContainsUnwantedMimeParts: N Alan Somers wrote:=0A= >On Fri, Sep 2, 2022 at 9:11 PM Rick Macklem wrote:= =0A= >>=0A= >> Hi,=0A= >>=0A= >> A recent discussion involving copy_file_range(2) performance=0A= >> included a suggestion that, maybe, copying of subranges=0A= >> should be done concurrently.=0A= >>=0A= >> Although I cannot be 100% sure, I think that this would=0A= >> involve using multiple kernel threads (taskqueue or similar)=0A= >> to issue I/O operations on the file system(s) for blocks=0A= >> (of f_iosize maybe?) concurrently, to improve performance.=0A= >>=0A= >> Doing this in a system call is unusual, to say the least but, then,=0A= >> copy_file_range(2) is an unusual system call to begin with.=0A= >>=0A= >> I have not attempted to code this up as of yet.=0A= >>=0A= >> So, what do others think of this idea?=0A= >>=0A= >> rick=0A= >=0A= >I'm skeptical. Is the intention to speed up copying on file systems=0A= >that do or don't have an efficient VOP_COPY_FILE_RANGE implementation?=0A= I suppose so. In particular, when the input and output files are on=0A= different file systems, a custom VOP_COPY_FILE_RANGE() cannot be used.=0A= =0A= > For those that don't, I don't see any point in trying to beat the=0A= >speed of the old cp(1). Apart from the problems that we've seen=0A= >around hole size, does the copy_file-range-enabled cp match the older=0A= >cp's performance?=0A= =0A= Well, the discussion starts here:=0A= https://lists.freebsd.org/archives/dev-commits-src-main/2022-August/009067.= html=0A= For some reason, there seems to be missing entries. I recall replying=0A= to the one that suggested concurrent I/O operations (by mav@, I think?)=0A= that I would post here asking about it. (I've cc'd mav@, in case he wishes= =0A= to comment further.)=0A= =0A= I do agree that doing some performance evaluation of cp(1) would be=0A= useful.=0A= --> The thread seemed to suggest (I'm no ZFS guy) that mmap'd=0A= copying does not help for ZFS and that doing copy_file_range(2)=0A= for small files instead of the mmap'd copying might make sense.=0A= --> Then there was mention of having copy_file_range(2) do concurrent=0A= copying of blocks, which precipitated the email.=0A= =0A= rick=0A= =0A=