From nobody Sat Sep 09 04:54:14 2023 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RjLHW1hkbz4sBZ6 for ; Sat, 9 Sep 2023 04:54:31 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic314-21.consmr.mail.gq1.yahoo.com (sonic314-21.consmr.mail.gq1.yahoo.com [98.137.69.84]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4RjLHV0Tjdz4DRG for ; Sat, 9 Sep 2023 04:54:29 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=EKK6YCv3; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.69.84 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1694235267; bh=v585CdV9aZ5BPDMDMynso/u18oN39eBJDpgvByZYHLs=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=EKK6YCv3/H8CY0qXxdj4WZb5KxrOVa7HCDz/uvU5cP8st+hR8TzF5IT98tKILQlXoYTMntjCDYRiWdTV3YthDsMJvrJ94tb3r7tp+JzA8n6NvLqlPCCLOb4E2uOCV6O136o9WNZvNnHiMrTraiL98it7iC1S/DyvcYW9CI3u53EXvMSwLa5LlWrD7dvjwcrVIXbe9vaUjBZcUYZ1MLoq4/E5aFllM1xX1yqUK4YoIHpxjRXHBnx1lCS26uSWiXpSzZrYVbm8YgEgXEQN1BjZmTbs1fd378ALpYX21QS+MLk8VMQGGV7p8AgJh9AwnPpEthyhIfPeFq2G7f1VYZ+/Ag== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1694235267; bh=lfgbNcpldc8ZauWVBpY7oLccS0XNDjvqHdWxUCn87Kl=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=p+Enj3LxwwfX2jAnzT1D3X6S8Lhb7uNt9odk7/78uY9/MiOruM9RGg7frEpBlp+TWv95MgkrN9Gh4G7fGVXxSvASPxt3jKUNuVPHmpvkr7qEvKuqF1LW2jnC14moLcRqRcthHWbBKApgx3Z8c4K+dGZ5ECHMjGGPiWncQA1NYuhNRGW0aIE/ngVlv2p+YFbnXDnsbx6Q0pXhSfG+/4mIuKg9oZB0pY/bLcOuFV0iK0WhcoFJKBtLykj3RJveO05Zi8ZmYwiCOObBMufeUhPgccb4mUBuqNIVO1PRWdg760S4xu1/27GNsvNP0L/uHCkHAb2XFNalvB3zXvO+n9NsTQ== X-YMail-OSG: p81Q6soVM1m_XWNTO7l1.pvAmHNtvdCdr8S1KM7O05wCtfGqc4x_gXZcbA_F5Rz bwn4cUBBh9Jqx.1cCPmrbkXOqsvglreqSa3FyyZ7dX.gXwuVLP5N9PCle2G0ikQVLTOY38UWUK9M GmuRthKtlSvqoo3dV3nomivLTvYWXJFB33rk69CeJXgXs2_2jE661t82dQQyO3HeY3G98egBEhR7 7aQnKhu159YZcrFfoEKyqOg1d5LSxd3vwJDraNn8Y0pOF.8aYEvePawTodAl4SK_1ZDWrQoPM9Md 7NGsWyFMleoPQcKZ1wTSRl5GwKrCig.GtLD7SpeAHj9ZazXhNiIiL_9pweIHhZGZA.tEQVnwxcsW SmJQ2cmAOy9PHKr0vKJVgdZ4EnYL2y5xvVyNq3RX8DU5wg7_reak1yW.m8mBAY0lKfJaUVt3mF3c JDkY5IYoFpjddspxR9r7dA.XEW7BA4L4uPmWHUb_F3K3kYefQ4e80t1zDqQ2f8ge4qiaCCrZ74lG 5SF1_uG.6.iSqHJxIAooPkYe3saHj642QSYho0BHyGgSbYi6qyweNHPs0G2.grW7DBW3Zx4PoIt1 TdS9DTo6S9tng_DfCft_7ZULL0oOzRp2NYBJfFbBcxJvzg9U54Bpj8rc3KkBBwqlmAyCrdP4l1_s oaPlq9gun_Me11hhA1KK9qGh5oqxoXrBC2eiGBOzXjbMZU3ItIci1nfSlHFH6uqvyE7Ryb_MFDnb tXXO8kiR.Lz.gD4fjB_aE02hd_nHFiQXdfRozvg2hmWk6QnmkqkbgtOxb08.XfVylldj3rBisMuB BnGu3tqc0ScEk2U5ZXUoN8mQBi39Ll3R6l7cKqNOXBZmS78aQvln7GvirfUY3oZOBugJLJZVa9qP a9rZLR5ZK1N5OTZyK5n6MNoi5fM1itDC7VHu.v2e5iMaz_r2ePA3km2wNK7O50ezJxlNtEFTo0H9 faxptq9UmT0qTE5NT6BVGjwDtlPgYjgsy7RV5CGpSkO052pM6EJSaUNCtTDDSse7nqAQi51GfzQt e9oJtairXriHKpSLiH.POTyuBH7zidOdIRloZnbT6uJ4UkRGdSQ5UJTPFnpyzmK9j8aSR34dZErG 5B0mYA6i.18vQz3rgPtY8junN4N208vwl2UF837qFt5kgCnY5N_62QDTBF5y86EsMz8vE9WpSMna DXSSJ_.CLccnt0PzQnZcBja_O5Kc79jlZNAgTT7XxuEo.bWhpn3w.qlRzymbwxKAs7Dso5L_KAIl gUAvQZH2Mhzp3miC30GG.Oz9gg43.ue0zF2hEaXbe73P33hnKXLZElLjtX0kdJcFtxvaXWYW5c2E .nGhHM7PjEVPtqLxvTAEcWSi7g0f.N6m7vUUSkLbrHfZJeff7m_h6o3v1lBTc43LEFTZGyRmWiRO XINiAtFgnusjcCzzsTPVr5LMZMoNkppiKdFgUeELUD73GLhTSYn_wubLxnuUu1gMJUsdJ1G0r09L ph5D3SwmdMFtGK_flMh_sKlsC94R1ZZEXKHgMBYxphaTSZ6FTng5UdjACsMRxSKNvIJxJr.XyhRk ZxWElNdWlTcoU6Zi6pDukLlXpu8g2NvfHm0E6oiC6EBfyDlWr.rOfKBfZHVsPoPTFTrn5tnjktgj br0Inv4NK_R3hXENa6vlkytIC1BkL0Y8DtrmtvCjBghi2dKUutrlY34jWs_FiResnZ4YyTEi5il0 UvIvXABOoIG0OD6WZn9Tys47R.XLCCdHq4xg7ebwmmO4r6yc9.kmRNnpwx7xFOQx73BPfj6qKrEg 1L1egrxbBKZ84bgkE9e6t5D_FcdArYruI8ZgiW1WdP3kiZF4jP.slaH6cbCDyi9sJkBG3w1NH425 qL8xRj1y0dSuUTPZfTF8AgDXarHux9lQVmc5e6csFVf7MVDeoiu75Z2nV4mEDszKBaWOt_6k87rZ aXdVapfgOsIpkX8h.wJo4.tMFALsS9BZbz5FixP.ZE1RkfFqkkIffoiMK92zuIrm_cISV_Wi33iv vm3aiplPAiRszqXeexpRDw0A9fnO220snMwZvVUFCCeAI6jO7UaBYfBfItwntZH_TXXGJmn9ilCy 7QGfrxBZ_nlTjVJ5QsXji4WNxpItWV1oCA2fe8EtuIK5M10VLvyKdKkXPd0UJ1rxZWt2tElzxveA qRO5ZmGafLZeC349Gn8nGchfE9Ad5TIeDX1HEyMsj0On4y_S0vfaRphDPVdhrUqXPr2EOgaEyQ7q qdYNiYUmJc3P8yVijjFmMK0sBv70dtL4JtDY.tQECX32hcPiuVa3QK9MoHGqjyCJAPg28UuRNsGk vbfd9mZEKLOs- X-Sonic-MF: X-Sonic-ID: 264c2783-4b2e-48d8-b189-d16e630b9fee Received: from sonic.gate.mail.ne1.yahoo.com by sonic314.consmr.mail.gq1.yahoo.com with HTTP; Sat, 9 Sep 2023 04:54:27 +0000 Received: by hermes--production-gq1-6b7c87dcf5-wlch2 (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 407869484ee6bdc0f2b208629e18a038; Sat, 09 Sep 2023 04:54:24 +0000 (UTC) Content-Type: text/plain; charset=us-ascii List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.700.6\)) Subject: Re: main [and, likely, stable/14]: do not set vfs.zfs.bclone_enabled=1 with that zpool feature enabled because it still leads to panics From: Mark Millard In-Reply-To: Date: Fri, 8 Sep 2023 21:54:14 -0700 Cc: Current FreeBSD , FreeBSD-STABLE Mailing List , Pawel Jakub Dawidek Content-Transfer-Encoding: quoted-printable Message-Id: <8746A218-F83A-40E7-95F8-5EC1E36411C1@yahoo.com> References: <7CE2CAAF-8BB0-4422-B194-4A6B0A4BC12C@yahoo.com> <08B7E72B-78F1-4ACA-B09D-E8C34BCE2335@yahoo.com> <20230907184823.GC4090@FreeBSD.org> <4f4e2b68-57e0-a475-e2bd-1f2b8844ebfe@FreeBSD.org> <354C5B8C-4216-4171-B8C2-8E827817F8E5@yahoo.com> <8B8B3707-4B37-4621-8124-D6A77CAF6879@yahoo.com> <15df58d3-4603-132f-112e-d10a6d4419bf@FreeBSD.org> <2a25427c-5a61-3f72-4e31-b7666741d38d@FreeBSD.org> <63717d32-f340-1320-3335-85135d1b62bc@FreeBSD.org> <05C47E15-640D-41AD-9C4C-73A1D5041CF4@yahoo.com> To: Martin Matuska , Alexander Motin , Glen Barber X-Mailer: Apple Mail (2.3731.700.6) X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.49 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.99)[-0.988]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; MIME_GOOD(-0.10)[text/plain]; FROM_HAS_DN(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MLMMJ_DEST(0.00)[freebsd-stable@freebsd.org]; ARC_NA(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[98.137.69.84:from]; BLOCKLISTDE_FAIL(0.00)[98.137.69.84:server fail]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.69.84:from]; TO_DN_ALL(0.00)[]; RCPT_COUNT_FIVE(0.00)[6]; FREEMAIL_FROM(0.00)[yahoo.com]; MID_RHS_MATCH_FROM(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; RCVD_TLS_LAST(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2] X-Rspamd-Queue-Id: 4RjLHV0Tjdz4DRG On Sep 8, 2023, at 18:19, Mark Millard wrote: > On Sep 8, 2023, at 17:03, Mark Millard wrote: >=20 >> On Sep 8, 2023, at 15:30, Martin Matuska wrote: >>=20 >>> I can confirm that the patch fixes the panic caused by the provided = script on my test systems. >>> Mark, would it be possible to try poudriere on your system with a = patched kernel? >>=20 >> . . . >>=20 >> On 9. 9. 2023 0:09, Alexander Motin wrote: >>> On 08.09.2023 09:52, Martin Matuska wrote: >>>> . . . >>>=20 >>> Thank you, Martin. I was able to reproduce the issue with your = script and found the cause. >>>=20 >>> I first though the issue is triggered by the `cp`, but it appeared = to be triggered by `cat`. It also got copy_file_range() support, but = later than `cp`. That is probably why it slipped through testing. This = patch fixes it for me: https://github.com/openzfs/zfs/pull/15251 . >>>=20 >>> Mark, could you please try the patch? >>=20 >> If all goes well, this will end up reporting that the >> poudriere bulk -a is still running but has gotten past, >> say, 320+ port->package builds finished (so: more than >> double observed so far for the panic context). Later >> would be a report with a larger figure. A normal run >> I might let go for 6000+ ports and 10 hr or so. >>=20 >> Notes as I go . . . >>=20 >> Patch applied, built, and installed to the test media. >> Also, booted: >>=20 >> # uname -apKU >> FreeBSD amd64-ZFS 15.0-CURRENT FreeBSD 15.0-CURRENT amd64 1500000 #75 = main-n265228-c9315099f69e-dirty: Thu Sep 7 13:28:47 PDT 2023 = root@amd64-ZFS:/usr/obj/BUILDs/main-amd64-dbg-clang/usr/main-src/amd64.amd= 64/sys/GENERIC-DBG amd64 amd64 1500000 1500000 >>=20 >> Note that this is with a debug kernel (-dbg- in path and -DBG in >> the GENERIC* name). Also, the vintage of what it is based on has: >>=20 >> git: 969071be938c - main - vfs: copy_file_range() between multiple = mountpoints of the same fs type >>=20 >> The usual sort of sequencing previously reported to get to this >> point. Media update starts with the rewind to the checkpoint in >> hopes of avoiding oddities from the later failure. >>=20 >> . . . : >>=20 >> [main-amd64-bulk_a-default] [2023-09-08_16h31m51s] [parallel_build:] = Queued: 34588 Built: 414 Failed: 0 Skipped: 39 Ignored: 335 = Fetched: 0 Tobuild: 33800 Time: 00:30:41 >>=20 >>=20 >> So 414 and and still building. >>=20 >> More later. (It may be a while.) >>=20 >=20 > [main-amd64-bulk_a-default] [2023-09-08_16h31m51s] [parallel_build:] = Queued: 34588 Built: 2013 Failed: 2 Skipped: 179 Ignored: 335 = Fetched: 0 Tobuild: 32059 Time: 01:42:47 >=20 > and still going. (FYI: The failures are expected.) >=20 > After a while I might stop it and start over with a non-debug > kernel installed instead. I did ^C after 2.5 hr (with 2447 built): ^C[02:30:05] Error: Signal SIGINT caught, cleaning up and exiting [main-amd64-bulk_a-default] [2023-09-08_16h31m51s] [sigint:] Queued: = 34588 Built: 2447 Failed: 5 Skipped: 226 Ignored: 335 Fetched: = 0 Tobuild: 31575 Time: 02:29:59 [02:30:05] Logs: = /usr/local/poudriere/data/logs/bulk/main-amd64-bulk_a-default/2023-09-08_1= 6h31m51s [02:30:05] Cleaning up [02:38:04] Unmounting file systems Exiting with status 1 I'll switch it over to a non-debug kernel and, probably, world and setup/run another test. . . . (time goes by) . . . Hmm. This did not get sent when I wrote the above. FYI, non-debug test status: [main-amd64-bulk_a-default] [2023-09-08_19h51m52s] [parallel_build:] = Queued: 34588 Built: 2547 Failed: 5 Skipped: 239 Ignored: 335 = Fetched: 0 Tobuild: 31462 Time: 01:59:58 I may let it run overnight. =3D=3D=3D Mark Millard marklmi at yahoo.com