From nobody Fri May 14 17:07:13 2021 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 55EC184714E for ; Fri, 14 May 2021 17:07:25 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic311-24.consmr.mail.gq1.yahoo.com (sonic311-24.consmr.mail.gq1.yahoo.com [98.137.65.205]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4FhZhX2h1yz4dtK for ; Fri, 14 May 2021 17:07:24 +0000 (UTC) (envelope-from marklmi@yahoo.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1621012042; bh=qbkPerKu1tuakm7eOtpu9Ld+r+oghoYyt6EkiLDac5A=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=e0n0esTSY1vYyqLQ/EEd1D8ll4Io1sWJzpS+3YhBSr8+ZzKKt0gWyxWvp9ijiLN/J9vo6JF1nkzVmlSntVGrzsCpbs97IfkWX2EbTg60+GenrvAg0t6Cn8smH0Im9bNN0pUicFaXmVpeVn959bzMmcNWGTthJ3sdDnpRzEtIl4uL3yHhwk2LSRUrbHKIB1xo+hVMIcfBYuRTgPGEOzl2/ZDPOPTrorpIXMx1bjep7x60C5fXl0bUEi2/fXSlExkFLeaXSfcttAUSsFRdUs+qVilT+2gIlcsFkK5SrAUtfMuwxTUzV3MFNMGRa7bfNJ7HwOIzUvkY3H+0INtM7tQITw== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1621012042; bh=HWgDaajK4J9i6os3iunJr4fbrEWEHC2Q16UlT4p6eqt=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=TdstHoMMwkhifoywhJxbWEWZ/VQxsstlofFDj2UcaIR+ekY1+ytaH9w+T14vdyw9jotdB+s/m4Ln5ctvRudb8Qz6avTyqJ2H6SunBmyphy3LTEnlFuY2eZ3611ZIanBu4rAVm+dY7CLEH4KE7Y3VC321NBWqROsKN4qE42+Z3GLrUTLZ0LmKvd2ptOTWFQHEH2Gd5CA4gcCYljsSZtXd5DjJqUj153ahymsAk2E4k+136kdKH/S1C2WAxNLq3WRlhhV5kvFjIOQtWbswaPy4zg3SqcGPVatgg8lDExMmdr6xx+XoXXF1U14yxSmS2/0ffoQ9iCbXK98V7CqbxddMCA== X-YMail-OSG: LZ9QDvwVM1lTmh5eErkm_h.FZpnNKn4iV9s9S04Wdazh7PbrH2vBbkQbnplOHpR RYX6fAPBybdZF7ANr6KIcht_Nj5bV27fhKNAFauFfgGPyoQq6dldROCQ0x0szNIVFzPwAAkHaNEO fq.BdOUUTia6PHbbB0MwPKrjjLaCkqg.K5eF99hObYvjZEmqgYyqUsaO6ZYTQHdeTRJ6ki2Gscp. 34axZPLLdF.9uwU553zFkVu8KZHbSiAGp6bERVKJdfYQ9oKULzboWbILIlR2AqmUZeFp9B0_HF1e hMFV9uZZYs722l.7HENS6h5umbsNTLygunmsE5jG.hs8S1YCmpYdBaK.2duN9.bV2Tvs4_N3utGD KGlOdzpVSjc4.JWdfQxBNPS.9G.XMnzP.bWFSdzYmIycrlgiHqaBAwvZWms0Cih0R_BAXTfa7h6C a0_HSh1GKwVVu99oy7G5YAn_Et8wtmh6hAFimRtkM3wysWw5bbJRnf9BogRbykVtvcTMKIca7Zij 9NRT7YXhAWk.gFOU_K1DZKL8DP7QJXrtSL97Lr.3IDkWkkhCD6F3mt2epFl.T4teCklszW_u1vP0 Orrvc.wfO.a3P9WnKTpG1_XsLHZoniBhX4HAiPlw3QHgQOtrlPLyrH_Ssks7M3qmWVBSy6otLlVx 8d.QPpN9mqkR_u0kAnA2RTl59E9817TDk.b1uaav9LMcqfBMyqMBTDu8WsiJSgoU7GwVnP9FAY5a 31Th4cK3Z7hClAibaYIn99odGy1kLHTYZaAO8Yk1IeWNQe..TBMVlV42qFQHH8DrWUd5kvNPdcnE Eajqbb.ZINRSMi2xnLxyC6WM8v7wDxbzc8AWmroJZbFPcnPaJa9.UIE.8XQUbEUF9flzDS9v2cBS D5gRQQYVay4K4flpVXErqeMTZ61QoSEQbU0jngRL2YnE2_N7QzSpBocrAWVjY9E7zXtzI9uNA3pl o.j9iyuvxxFiO4G.n1E.xP0Vo6FFErW97Sz9CTwUORL0YFnlp3V5ksqXCiv5mfZUjOhHsJrqXgWv cVsfWKVKQrp5M7j4Xdy1IKYyRoCy0_m65ozqEnAQsgYM3jfeytEIKvmhmMlNMU2UQpeKaDmWw9TT RlI5vzBCF9TSyfW7huWLtpFDiOf89yhPNj4ZQB_qJgrkDNEouFa.VsYpKo.ZftffQG15DcJNSUOu jRm4EYOdRImhoa64ABYjmSJJ1g0mP11bzQq7sBlYHfuJVJR9w2ViaUyBk1228pLcG0UeiYUoYueT d9j3ZiRcsgL9UVb1pxvY8rYbxak5HHYxUmgrYIHk3Hv6xYWLgB0FDoiwW1WUGHAE1TIxaiakp0xG x3m0HBRkteR.nsN4ujAOxGQe7MjE.Tcl2f7nU7mTxIW6SUFbHEiEGH33Q8irrVqqn.21HcJHbZMH fPoNPrRoJmMIMlb04iae4aWqtZ.JlMr9..XuN72.xHPbkrlliMWtxomTMaaiABgxRV4f82VOzzCJ 8IMTSv4uqpKpyfLscYMI0_XHy1WU_ReZ2tHLOGxQKPizcyfVKKzzJWmJMDuCIom_7G0bkvDTLtla nQwaDXvjAsKcX0HZhwi3rXBQLxZYhhZQZV2ZwIgUxvKTrX8QCH25sXGgim8JoJJV8YaIPHYdf7I5 izg8v8zYxA4tuQx0ETn96JDDr52D0.FxokdlEUNhipPUHKOJq44aZ3ptQvThOR5LznIBLL8iDBTy .mINkr6u5bjkAcgKyr8oHkQC1sv4KEhGui0f3DzB3zqdyjjNkC7YpFQ96OI56J9uLCHFRGZ72ueN 1rZKlA_3g0uB8pGvDsZcaxvEhETHBBOzSnrgDCsxSdMbBRqDT_pcQM9uOqj3zzAVD6zfLFeSDKHy tnuUvpRccbRzM6KtI59EZaQ.S4b.KEmLPudeCtgNCLgd8VOcIHLXNjRGcncIPGsJPAWCWmVusBhv VAFiqhCip.YwKU1MQ_K6nBTp.wsJ4mzzmlElFseAAIwt8hnMoy4vI.fjAqZ2.xdI6gjm2NIH4iRY q6VIWMd0QkY2gfSUbXAf1za658sMRAA0Pj7aStARGl.N17AZ6q8Vy2OEYRVYwGCA6yLsrnW_gAVo 1URiEvifgr0fenHLcCsuAoQ5zcXPgcZAlTUkpdfZd5QQrN.QfPl9a76fuDu2RU7QmIMQ_8ypJwqk J.Q4fAl5i39t8BBiENqrN73gDOSc7PsK7nAiw1QNYyvZItEgpPUjmBd.jNK4QinCW4BN2BT3ZElp LPHxhUTCAEjzvC.L3lwXAPEUQrA.OOVgoDphGGIoU1erycJwUYKGl3tdWX5aYr8AQqfDHStY7DT4 pAqa0Oy89OyNwdaCyORqZqaEZZq5R_lA3kAzEknZ0oGSicCVuMMhkMrPOI7cTJ4uRtFXEh4DCBkZ juYt2DdregI8bjsyt5ctLoc5ShJvsiq9QsOnumZmJPmTHDwFo2OgCAitUh1C0BTnibLmTIFQHLHk 1lDNhz0wbVTkH9tHd8HRo8QqrjlmnHTLdufzt5CSJLMzkE49gmB9OhmhSgnZUdvbtSE0y.wHpDbp vddEd1sgOUDRnAEDaGEc5xdRYN.y5MdZBtskTxRZ5GHKH._EzUMZkvxbgkYgIMWBwFKobc158Fw4 Vr2KQIrzBa_IdKP6sR.D5vFZZjU6_2QRR4Z989rMq40Xz6qPZc2LPagmWp1DSIfUYqauul1HYUJ7 hGgneOB3ofs0r78JfjP_4rr_MGchJyd.LvhY_7Q-- X-Sonic-MF: Received: from sonic.gate.mail.ne1.yahoo.com by sonic311.consmr.mail.gq1.yahoo.com with HTTP; Fri, 14 May 2021 17:07:22 +0000 Received: by kubenode550.mail-prod1.omega.bf1.yahoo.com (VZM Hermes SMTP Server) with ESMTPA ID 6893f48c18eaf5d8c0920d8e64ff2b16; Fri, 14 May 2021 17:07:16 +0000 (UTC) Content-Type: text/plain; charset=us-ascii List-Id: Porting FreeBSD to ARM processors List-Archive: http://lists.freebsd.org/arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.80.0.2.43\)) Subject: Re: FYI for aarch64 main [14] running a mid March version: I ended up with [usb{usbus2}] stuck at (near) 100% cpu In-Reply-To: <202105141252.14ECqA0h081135@gndrsh.dnsmgr.net> Date: Fri, 14 May 2021 10:07:13 -0700 Cc: freebsd-arm , freebsd-current Content-Transfer-Encoding: quoted-printable Message-Id: <14C02B9D-7342-41EF-AADD-E466C04E17EA@yahoo.com> References: <202105141252.14ECqA0h081135@gndrsh.dnsmgr.net> To: "Rodney W. Grimes" X-Mailer: Apple Mail (2.3654.80.0.2.43) X-Rspamd-Queue-Id: 4FhZhX2h1yz4dtK X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=e0n0esTS; dmarc=pass (policy=reject) header.from=yahoo.com; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.65.205 as permitted sender) smtp.mailfrom=marklmi@yahoo.com X-Spamd-Result: default: False [-2.14 / 15.00]; FREEMAIL_FROM(0.00)[yahoo.com]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; NEURAL_HAM_SHORT(-1.00)[-0.996]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; ARC_NA(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[98.137.65.205:from]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; NEURAL_SPAM_MEDIUM(0.36)[0.361]; SPAMHAUS_ZRD(0.00)[98.137.65.205:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[98.137.65.205:from]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.65.205:from]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-arm] Reply-To: marklmi@yahoo.com From: Mark Millard via freebsd-arm X-Original-From: Mark Millard On 2021-May-14, at 05:52, Rodney W. Grimes wrote: >> Note: The context was using a non-debug main build >> from mid-2021-Mar. (More details identified >> later.) >>=20 >> The issue happend while attempting a: >>=20 >> # zfs send -R zpold@for-copy | zfs recv -Fdv zpnew >>=20 >> where the drives involved in the command were: >>=20 >> zpold: a USB3 SSD, using /dev/da0p3 >> zpnew: an 480 GiByte Optane in the PCIe slot, using /dev/nda0p3 >>=20 >> with: >>=20 >> # gpart show -pl >> =3D> 40 468862048 da0 GPT (224G) >> 40 532480 da0p1 4C8GCA72EFI (260M) >> 532520 2008 - free - (1.0M) >> 534528 29360128 da0p2 4C8GCA72swp14 (14G) >> 29894656 4194304 - free - (2.0G) >> 34088960 33554432 da0p4 4C8GCA72swp16 (16G) >> 67643392 401217536 da0p3 4C8GCA72zfs (191G) >> 468860928 1160 - free - (580K) >>=20 >> =3D> 40 2000409184 ada0 GPT (954G) >> 40 409600 ada0p1 (null) (200M) >> 409640 1740636160 ada0p2 FBSDmacchroot (830G) >> 1741045800 58720256 ada0p3 FBSDmacchswp0 (28G) >> 1799766056 176160768 ada0p4 FBSDmacchswp1 (84G) >> 1975926824 24482400 - free - (12G) >>=20 >> =3D> 40 937703008 nda0 GPT (447G) >> 40 532480 nda0p1 CA72opt0EFI (260M) >> 532520 2008 - free - (1.0M) >> 534528 117440512 nda0p2 CA72opt0swp56 (56G) >> 117975040 16777216 - free - (8.0G) >> 134752256 134217728 nda0p4 CA72opt0swp64 (64G) >> 268969984 668731392 nda0p3 CA72opt0zfs (319G) >> 937701376 1672 - free - (836K) >>=20 >> The system running was that on /dev/ada0p2 (FBSDmacchroot, >> which is UFS instead of ZFS). >>=20 >> The [usb{usbus2}] process eventually got stuck-busy, no >> more I/O: >>=20 >> CPU 0: 0.0% user, 0.0% nice, 100% system, 0.0% interrupt, 0.0% = idle >> CPU 1: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% = idle >> CPU 2: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% = idle >> CPU 3: 0.4% user, 0.0% nice, 0.0% system, 0.0% interrupt, 99.6% = idle >>=20 >> PID USERNAME PRI NICE SIZE RES STATE C TIME = CPU COMMAND >> 15 root -72 - 0B 262144B CPU0 0 8:51 = 99.95% [usb{usbus2}] >>=20 >> 1295 root -8 0 20108Ki 8092Ki q->bq_ 2 0:04 = 0.00% zfs recv -Fdv zpnew{receive_writer_thre} >> 1295 root 48 0 20108Ki 8092Ki piperd 2 0:22 = 0.00% zfs recv -Fdv zpnew{zfs} >> 1294 root -8 0 17544Ki 7740Ki q->bq_ 2 0:01 = 0.00% zfs send -R zpold@for-copy{send_reader_thread} >> 1294 root -8 0 17544Ki 7740Ki q->bq_ 0 0:00 = 0.00% zfs send -R zpold@for-copy{send_merge_thread} >> 1294 root -8 0 17544Ki 7740Ki hdr->b 2 0:00 = 0.00% zfs send -R zpold@for-copy{send_traverse_threa} >> 1294 root 52 0 17544Ki 7740Ki range- 3 0:20 = 0.00% zfs send -R zpold@for-copy{zfs} >>=20 >> 1036 root -8 - 0B 1488Ki t->zth 0 0:00 = 0.00% [zfskern{z_checkpoint_discar}] >> 1036 root -8 - 0B 1488Ki t->zth 1 0:00 = 0.00% [zfskern{z_livelist_condense}] >> 1036 root -8 - 0B 1488Ki t->zth 2 0:00 = 0.00% [zfskern{z_livelist_destroy}] >> 1036 root -8 - 0B 1488Ki t->zth 1 0:00 = 0.00% [zfskern{z_indirect_condense}] >> 1036 root -8 - 0B 1488Ki mmp->m 3 0:00 = 0.00% [zfskern{mmp_thread_enter}] >> 1036 root -8 - 0B 1488Ki tx->tx 1 0:00 = 0.00% [zfskern{txg_thread_enter}] >> 1036 root -8 - 0B 1488Ki tx->tx 2 0:00 = 0.00% [zfskern{txg_thread_enter}] >>=20 >> I was unable to ^c or ^z the process where I >> typed the command. I eventually stopped the >> system with "shutdown -p now" from a ssh >> session (that had already been in place). >=20 > Should this occur again before doing the shutdown run a > zpool status & > I have gotten in this state when the recv pool was a usb device The USB device had the send pool in my example. > and for some reason it had a timeout and gone offline. No messages about timeouts or other such were made. > The clue > this occured are in dmesg, and zpool status. No console, dmesg -a, or /var/log/messages output were generated. (And the system was running from a SATA SSD that was operating well.) For reference, the USB EtherNet device that was in a USB2 port continued to operate just fine, allowing the use of existing ssh sessions that were displaying gstat -spod and top -Samio -ototal until I started looking at the problem. (I did not try making a new ssh session). I did not do the "zpool status" so I can not report about it. > Unplug/plug the USB device, check dmesg that it came online, > and do a zpool clear. Okay. I normally avoid unplugging USB storage media if the system overall does not hang up: hopes of a clean shutdown leaving things better. The system did appear to shutdown to completion. >>=20 >> When I retried after rebooting and scrubbing (no >> problems found), the problem did not repeat. >>=20 >> I do not have more information nor a way to repeat >> the problem on demand, unfortunately. >>=20 >> Details of the vintage of the system software and >> such: >>=20 >> # ~/fbsd-based-on-what-freebsd-main.sh=20 >> FreeBSD FBSDmacch 14.0-CURRENT FreeBSD 14.0-CURRENT = mm-src-n245445-def0058cc690 GENERIC-NODBG arm64 aarch64 1400005 1400005 >> def0058cc690 (HEAD -> mm-src) mm-src snapshot for mm's patched build = in git context. >> merge-base: 7381bbee29df959e88ec59866cf2878263e7f3b2 >> merge-base: CommitDate: 2021-03-12 20:29:42 +0000 >> 7381bbee29df (freebsd/main, freebsd/HEAD, pure-src, main) cam: Run = all XPT_ASYNC ccbs in a dedicated thread >> n245444 (--first-parent --count for merge-base) >>=20 >> The system was a MACCHIATObin Double Shot. >=20 =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar)