From nobody Tue Aug 22 18:24:00 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RVd6G3kMZz4rDrX for ; Tue, 22 Aug 2023 18:24:22 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic309-21.consmr.mail.gq1.yahoo.com (sonic309-21.consmr.mail.gq1.yahoo.com [98.137.65.147]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4RVd6D4Sgqz3QkY for ; Tue, 22 Aug 2023 18:24:20 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=pK0OCuEi; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.65.147 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1692728658; bh=6dLtdxIx1FV+bYeMYsCQIZSgbgzUJPb4BrLA/Z6ny9w=; h=From:Subject:Date:To:References:From:Subject:Reply-To; b=pK0OCuEi/lZgrRDg5u6VyeJPjjstTyc0Y3hSKrwmZKHDeghewWB9TnlGRRtcMiHHFKRJ/nh7EzXF2WbeuUD7cQc2TfGYqSaGfcngTAIG8rQ1dJ3OhROONeKd+9GPkqot+DMhW5uRu0IBDPuepQRBOXAzYkYKxncM1/ARoT5MyDGvKHJtFg9uSWpNyuzkfhZzqbHzqd90HZ+B9mVWAVOA6xmJUoVxnulcNjEtrt+KXYW99cYvNrrnXvAzcrKrS8ck+wUzEgFYCfXGf+R/4OvM9Zr8He6VkzGPYBK6godGeNtKTpm3deZLhB+bFmU+OQxmu+qnJ9aagsdHu+aO9jnySA== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1692728658; bh=1a3RdOUFXCuJRGajKYpNOKo5cvgLZlX0eaQM8Iw0Iz5=; h=X-Sonic-MF:From:Subject:Date:To:From:Subject; b=SeYpbS2DQkqWjJomYBa3X3mauh4EoRLVp2dxkZoUI0N5aNNY+WIPr7fUpPWKg+jPaMAGZFd+xjAVbzcBZhb6cy/dPosicjGraQd1sV4P4TKjaSUZ38VjQknbDeWNtth8mQZE9fEu9OKejCsQ63ClcpQleEiFp8bPekeLRT6fdzBgyU3Yl8biMbFD0LK+LguJBFVKmBbmSwAJEUwNu9cqGOT1JVoMS5W0FOweEKqdf/v7+HE2eMLAKGglV7xoRNqNNSDJaUMxSgX3XovR8amwti3GhDB80l4QUhcYMuTpGb/0R2oRj3Q5vp0j5Z9apI5Ey7zyIWQrWS2W45YWv63WeA== X-YMail-OSG: B02N1HwVM1mh1cvRVMLMdOfmWs5gwkG8fGhjWL3mIyiodVODJyQTCnuoPTKIQ66 4o1C4VZojLR4bmz1EVft8tmU.v6XbwDHMv0pKo6kp1TeoJq6tHYIp28TqyO5HapH_1v5tSKeRJjJ I2BcOl81vaJomjQDXQw9w8P.U2XgzMCHTK9T_0TgxppkPLgsoUmFr3KrH1XeVURIXOjfHRvPB32N iU2vqkvButyQtqu1KATWqljEqolapVOUfLpTpO3O9_AsSTmvltfJpnB9UlTDBulEKdJ4K64.oE96 LyEikKMJ63rYiOnRbhSwq1HjLzZs4JC2Lt7bqBVb_eitaqKv.gpEnCzFRLBE65G2ZDslyb48S8Ke a8W8cdjqNz_AVJll53bFfr8a5b8K9TZz2hziQZm_DM_nDq6q1S0ZSdddrbLbIIPdZJKZ7YfGRWwc uRE7f.EvITPq_DH29J4vPQE6HGj5ygOvUOy57wsrcZvBzUN8blIjQaOcfhGE0OBZNt2jdkzq9TN. 8_Hk60Q6ka2I9xoGfo5clsXHTtIF4JhYZE4YvUNjbrpHgxfy_BfQxYXwlsrtGsZ9wV1ZNY4gOwMG sPi_aMWUCiZhNL47yyeMHPcn7y6bvC2oTMImLiMMXDCJ.44ccm6FXDKHq1riOrHSjF.bA4RwVvg6 iS6vq2Eb_ONEid_jbgeZK9tfAE3YcdrkHq4FMmPdWPf7axIbWXYHFEyPd852In6HUjSNFW3XRkAW Gk6UH3T0mEIJkjyXhfqm2qd0zprVQfvZ56HK5zAlBsAHQramzpNEFKXgqH4kgMftIzbmawOTsxK6 cH4zDmmwwsBiQDg2ACIhi9krAYbVguiUrvQguu959WeRG.ZCO9DVlK3SH1St0k3A1xZcpvXpqUz_ sJmQm9GrkA8Tki5qc_osU78_gtG1EzGRN9GFcD1h2wOZWXEPOR4Lc9fz6ZVe60QF9pqfGLlsn1fh tL5UuxbRL_jV6JvQuXIjnNwxfi97xnAQi1vNyCGVIRQrfsnX1tgBPH1jwTWUqKtRqrkmTZr1lZLP YDqqtyRgGBQ0u0Sh87Ss7JOw9X5L2h8YSp3MjC0CGJTttTnNEZHrAkiwrLu3HocNCXZ7Oqbgf4kE lSuF13AxHCtJNQKQQowwAej18q6YPZ6ytgtWk3LOjcokJ5aK9sn6Fz6j7bWUrMGX_DjqkU9TaTGb PX.y7ftd9J7vrqnrykV_mxXPVD.rxuCSUomCwtqPRJGBvwNn2ho8gohzEf2_v.RxmoWU0MnOgUGK jdUhUuz4XCJZjN4bkneYJ9dc9XIGpcmeCKEvzJz_8fHSc6uunrV1S9sVrdo5CZK29ihfa9yzxyKM JJtRGkxFAQDmJdkeq8o5VolHibM3giGZeY3LOezVyDUVtO1qUxRaI75ccglXYkXHZ7W2YZCwQIGm iZkKsooO85zbJ4ncSOLrfRbe0CwvXIsIlIMhk1b6_IGN.b3BOhlAMG95Xlc7p9KAM03OHhW2p01m 6JNyPTsObVSX9Ot9K6455XIKJeEoxDWqDZx8TRLpz_o1dyvvJ7RqSIzex7ekbzsjMDNRJwrZpMDl a88AAL2ugNVwbS9b.YEMZvjD_SrYtiIV6OMk9rssow_rz0YNU_I3tlhnDWHW0sSxaIL6up6prLSW mkuGLTyzadbw4MgOp98uTUG7hlnZWA_BuNfsJoLBYAyZBA8K8QqI1KReW5.e9XCrG_nsNJtFahHG F40kpyKkWoP8u26I4iLnBtWt_I2QCl.0K6prPp14acHneA8Qt.d7UO6bcaVTaPYE3479C6XjJSCK DlXJHB.U2eruAzBq_de4bo9DKmpywnbL04vgrzofVMp7GL0cgbXrA9g_jDXrEpK30nQmvOu7jHfo sJlSjBhzrmOafvT9gHbpDVSH4E7DciFdOic2PcxDJU040HRzsh2rHxhsvTVQ5z4Fcs9sQsdNvuM6 Xkj9xg9YmjHbNn.1hGMJvcsR49Qi8OYobQCxN5_wXcJwp2Y3D16acPMTaP.Gu7XcAdo2txRQ8VNA fJcS7RcwkWrdrzKXNSByl3sofvYxlf47Kf.izcF0jPAVyyN4WxwAfW2TMAeJ.B0g1lyKHvgFbq9E NH0mCCuWh5CWUptW66t24xyPpxT7xqUtenR5dcMKS7uAD1mRf6Df91O5A.pi5Kze7yzlALPaStFE 7SNxD0DJ_kJNpw7oiZUY.FixbaUgSr2TpQEsfhc9uuCaWJavDEWCvPUaQdUp0I8rxag_ekQpBR9D ptmaezBj0nCHh__5V1m0G.KfVLkLGmaQ.hrS94kVcICVOLY7DYMbQUa4ALV819RS7q1DIpD4- X-Sonic-MF: X-Sonic-ID: d7414c24-6a7b-494c-9860-274f8e25d276 Received: from sonic.gate.mail.ne1.yahoo.com by sonic309.consmr.mail.gq1.yahoo.com with HTTP; Tue, 22 Aug 2023 18:24:18 +0000 Received: by hermes--production-bf1-865889d799-7x4p2 (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 142457281a2c78487630c2b1feea3719; Tue, 22 Aug 2023 18:24:12 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.700.6\)) Subject: Re: ZFS deadlock in 14 Message-Id: <4FFAE432-21FE-4462-9162-9CC30A5D470A@yahoo.com> Date: Tue, 22 Aug 2023 11:24:00 -0700 To: Alexander Motin , Current FreeBSD X-Mailer: Apple Mail (2.3731.700.6) References: <4FFAE432-21FE-4462-9162-9CC30A5D470A.ref@yahoo.com> X-Spamd-Result: default: False [-3.50 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.999]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; MIME_GOOD(-0.10)[text/plain]; FROM_HAS_DN(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; ARC_NA(0.00)[]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; RCVD_IN_DNSWL_NONE(0.00)[98.137.65.147:from]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.65.147:from]; DKIM_TRACE(0.00)[yahoo.com:+]; TO_DN_ALL(0.00)[]; FREEMAIL_FROM(0.00)[yahoo.com]; MID_RHS_MATCH_FROM(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; RCVD_TLS_LAST(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2] X-Spamd-Bar: --- X-Rspamd-Queue-Id: 4RVd6D4Sgqz3QkY Alexander Motin wrote on Date: Tue, 22 Aug 2023 16:18:12 UTC : > I am waiting for final test results from George Wilson and then will=20= > request quick merge of both to zfs-2.2-release branch. Unfortunately=20= > there are still not many reviewers for the PR, since the code is not=20= > trivial, but at least with the test reports Brian Behlendorf and Mark=20= > Maybee seem to be OK to merge the two PRs into 2.2. If somebody else=20= > have tested and/or reviewed the PR, you may comment on it. I had written to the list that when I tried to test the system doing poudriere builds (initially with your patches) using USE_TMPFS=3Dno so that zfs had to deal with all the file I/O, I instead got only one builder that ended up active, the others never reaching "Builder started": [00:01:34] [01] [00:00:00] Builder starting [00:01:57] [01] [00:00:23] Builder started [00:01:57] [01] [00:00:00] Building ports-mgmt/pkg | pkg-1.20.4 [00:03:09] [01] [00:01:12] Finished ports-mgmt/pkg | pkg-1.20.4: Success [00:03:21] [01] [00:00:00] Building print/indexinfo | indexinfo-0.3.1 [00:03:21] [02] [00:00:00] Builder starting [00:03:21] [03] [00:00:00] Builder starting [00:03:21] [04] [00:00:00] Builder starting [00:03:21] [05] [00:00:00] Builder starting [00:03:21] [06] [00:00:00] Builder starting [00:03:21] [07] [00:00:00] Builder starting [00:03:22] [08] [00:00:00] Builder starting [00:03:22] [09] [00:00:00] Builder starting [00:03:22] [10] [00:00:00] Builder starting [00:03:22] [11] [00:00:00] Builder starting [00:03:22] [12] [00:00:00] Builder starting [00:03:22] [13] [00:00:00] Builder starting [00:03:22] [14] [00:00:00] Builder starting [00:03:22] [15] [00:00:00] Builder starting [00:03:22] [16] [00:00:00] Builder starting [00:03:22] [17] [00:00:00] Builder starting [00:03:22] [18] [00:00:00] Builder starting [00:03:22] [19] [00:00:00] Builder starting [00:03:22] [20] [00:00:00] Builder starting [00:03:22] [21] [00:00:00] Builder starting [00:03:22] [22] [00:00:00] Builder starting [00:03:22] [23] [00:00:00] Builder starting [00:03:22] [24] [00:00:00] Builder starting [00:03:22] [25] [00:00:00] Builder starting [00:03:22] [26] [00:00:00] Builder starting [00:03:22] [27] [00:00:00] Builder starting [00:03:22] [28] [00:00:00] Builder starting [00:03:22] [29] [00:00:00] Builder starting [00:03:22] [30] [00:00:00] Builder starting [00:03:22] [31] [00:00:00] Builder starting [00:03:22] [32] [00:00:00] Builder starting [00:03:30] [01] [00:00:09] Finished print/indexinfo | indexinfo-0.3.1: = Success [00:03:31] [01] [00:00:00] Building devel/gettext-runtime | = gettext-runtime-0.22 . . . Top was showing lots of "vlruwk" for the cpdup's. For example: . . . 362 0 root 40 0 27076Ki 13776Ki CPU19 19 4:23 = 0.00% cpdup -i0 -o ref 32 349 0 root 53 0 27076Ki 13776Ki vlruwk 22 4:20 = 0.01% cpdup -i0 -o ref 31 328 0 root 68 0 27076Ki 13804Ki vlruwk 8 4:30 = 0.01% cpdup -i0 -o ref 30 304 0 root 37 0 27076Ki 13792Ki vlruwk 6 4:18 = 0.01% cpdup -i0 -o ref 29 282 0 root 42 0 33220Ki 13956Ki vlruwk 8 4:33 = 0.01% cpdup -i0 -o ref 28 242 0 root 56 0 27076Ki 13796Ki vlruwk 4 4:28 = 0.00% cpdup -i0 -o ref 27 . . . But those processes did show CPU?? on occasion, as well as *vnode less often. None of the cpdup's was stuck in Removing your patches did not change the behavior. So far I've not seen any similar reports to these resuls that I got the ThreadRipper 1950X that I have access to. I normally use USE_TMPFS=3Dall but that hides the problem and is why I've no clue when the behavior would have started if I'd been using USE_TMPFS=3Dno instead. I never got so far as testing for the kinds of reports I've seen about the deadlock issue. No one has commented one what I reported or if they have done any USE_TMPFS=3Dno style of testing. (I also use ALLOW_MAKE_JOBS=3Dyes .) The ZFS context is a simple single partition context. I use ZFS in order to use bectl BE's, not other reasons. =3D=3D=3D Mark Millard marklmi at yahoo.com