From nobody Thu Jul 06 06:42:08 2023 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4QxRly4hjQz4lvBF for ; Thu, 6 Jul 2023 06:42:22 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic303-23.consmr.mail.gq1.yahoo.com (sonic303-23.consmr.mail.gq1.yahoo.com [98.137.64.204]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4QxRlx4hhvz4G2g for ; Thu, 6 Jul 2023 06:42:21 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b="qJ/Jis/z"; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.64.204 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1688625740; bh=50gQWa7s7D4otG2q3WWzRxIoluupna6seu1NCgR5/Cw=; h=From:Subject:Date:References:To:In-Reply-To:From:Subject:Reply-To; b=qJ/Jis/z2DtJrIAWlKkjW4ehTyi4S2W0Ep0VuxO1bFAlkftX6LpwTTQTdVG2Y+GmpPUyYneqizoa7llbYyhS8homO/SfPsPwr084yMuY/0f/qoumiBiQHQCUSX3SbOz2SAer1JPr/rGmMqK3BxhoXzLNJScnxiSM38OtgjPcLQGP+gz+C2mE18/OIVPx6ZazXxnrDMk70MO8SSVK8RKTd/WBqmqRz4fdDPDLKLiEqDiwWQ2JoTpptlyOzoMdUxLIXSEkYURRS+4z0uiQaYh/17nymZrM2iWD+3HmAsKDrM/NieZdW43HH4bsh6IPyQwgr8VoBE+Er2vm2yPYGhNH7A== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1688625740; bh=raRRV8VzdTZd8bSbh8INAkRC4cJaq62kYm/DgmAwhc9=; h=X-Sonic-MF:From:Subject:Date:To:From:Subject; b=VbUmsHlIVve3DFpposJUYqV8syzo7XlKgV1Ctt3sgrct8MJgWGzvHKlKxVRPOWZWHssGstPCxkLzJrX0U256RPX9Lkhwmylf71JVeKKEwXV7PbW+OKRMXitrA3pXiBjtjmqRthOF6P2ivkfq6MBlXSyikOloAeaqpR1A5W9Gvv/pjsFAToexoT7muiaKAob+/FWBnSeNbHqR1IvAtun7+HLAQTGs8ZKWnD4ydd5nezHX4tKmMCIMhUddsArbBL009CRzDhN/6X32GDSu1xgUhQeQBp5foH4c2EfDDlC8fpwLUfJJ76JvS4iMGbDyohQ+925Um7VEVqcXCM8PeiLxfQ== X-YMail-OSG: r11b2U8VM1l90_9xup.tLRUMhEO1JZecZtf04o5Uj7Q5KkK0NP3u4tMZitJdsWq 4iFJDB8UC9ctwH8rpcE8t2FBworgBBmHeUnWmZqPZDQodvF1BSktMomsy0E6.n.QyZqOWKu1vRlh Yx8Vsq0ZnUek0rm6FJigRdzLG5G91bEPHi2vXm9Zzfev9FdMjTI_rmAjL46RepVfGGZNCL49Na8U x8rYXRdVgkU..JrfV96H6enIZuQvEo.4ZyhsILR6X4IEyWS0w_dooVzqbNoTIPnhBdVIjaZ932uF R5EvWtMXPD2vbV3jw6FhSIhHsvHxUf.zvnz_shne9COFH4Tl9Q4I4Ecomgh3HYuNMu8LAmoB2BNO sK954XbZEOZdLApnvGYyccjKV5kiIz.gQhfxadSIVLMATias_pSr7UlC50P50gEC8eNGiv.bc_y. Zm270M._4qwOElTwatcHjn84VKXfsK_P5wa2Ky4LBCjQUJ200.8MRnCO7XT4PPWxsG16cfAqxahP kAdQsn6kxoPHDLZrUTdRCL_UmssK_7Lu5F5ekuMDP9vCATCSK1YAiM9W.r6t2AmepgsPkqsVLaI2 x.z0i2tTLYK6knGV5dmn4JeScLFnzyGrN079OKlqvh4lX3JzL.CTu5_yjOE1VQmgS6Ke02BQuY6P W1yzwxrPKUocinEIallw6QqeAeiGL5L9rr.D_1ZHpaN5xaaoPyUW4vehOiCF83ZnlrnBW6ThdtJD sZztZzstG965PB.Ho_qyRtEXlGsdJWkJ6Bgi7ioi6a5sFmQsxaemXE0vdsTcsqFsSSg2PCIEcxz9 68UlNyH3v31gQOX0beXQjhmGFCaIoDauyiOSXu1NdHbelkYvFuvsP0XtAgni.ZofXqYZyAQUXWhA IZ7q270gGRHHNg3me2d9B1YdyARw5iVtHF6ZNjamHGrLGhiyEPKqlqJLe9ZRla33ckhfloNJjOxD ygIWw4n3upoxUv5SZArWiXUhZKXicNsA.KSaGeRl1d19Z6i_bPD05r1V7BC8p_GGim.q7oNGTQTv xvpGKW2pB7okq5ZTT3NZ7bYVuGFL08OvOomRBOZKwOdgpQmBBHZvo9JH4LojIypDnUE.XD_p5Qkb fZOyP6BeXTrG2rmW3ZhPfccIZR8L4kcBUmJgAUVvZNJdkFvZbjauQHnZ5CE1ZSZR.xUDNXZI899R TZl68rqKvyjqtof8sfbxTuCIzgzH32sg7CGBXOLjINPpzdI0iA2Ski3nkZ0KThN7lrCv3CvkFztz CCTF3xVfP8Y_2jThiP5Y4eqvbtuRNXAB4f4xFCHkhF53ES.Zh1XSeZWgfckA2mK09HbBZPZKNTeJ 4G_35miOO35RZ7wsEyQEEjeYrjurb_tN5R9ML6KquwVzBk5FhVvlJ8Ufr8mOfceNGAglSLqPwKJv aGO1_OcQrFb0dwnEv.cN99pDX63k1PZsRr0AG_THFMAm1BOkADQHuzd6wu7LOb.ZP_h0Ea45_ios jmdVLrtvt2MBevW7_ra5oTNmEwpkqFB6m_Y_pYDtJ0HpE2wDyhCqMqUJFSBHzAQ3rnc2hJfIcMd4 qn6CSuMFt2fhqB6ka5xNVX8JYZAyEqO9DJeisJNFo3y6_vDwDI7bN_zwWxRGjaqTs7y1Hi_uZYNs yxGu0GCkMCZRhQ07oKZiK0NH1JLcWVqacHGTYobJZRhc8RbT2_q.De.D46qDlkCnK3RdxTPA3oH3 XOVdi.9jjznb5mpPGbokDPw4332d7o6_CGYHBtzE2MsH0qurwkgivCP8aEeGB7drYdzmTyuVfvQ1 uyNd9Nau_otHdMUSfVaA_50TQ6_qyotGyYbOgMTO2JlAbo8jntHF3Ir1qLjzPbfe.VkQSCgx9dE7 N8xLZ2PPW766CnSk1Dt1.z4bLMDcS7rjX.eAHNrjx.IyP6qhkVIVHv8lzLJlIYw0B9unPsBJhEQh PGoBgcxEoMnRbg2sPPt7giV6alOwR.V0le8S9HF1nFgQefJR8UWABLIUqV3A1NHn1m5dRIj4aitL 73tYhhV_jcQNwN3UYZRxdWdnk8U5HnVzh_LaLwVBZPpyVjIsNfiOC52BnDyJriqnFJWRXAlVPsCf BohSbIjE5A4hH_XLDQwSy1452Ds18P3hQ3GdPQwo7MLJ6ytkr1EfHD0PB0YlzbsLtwVNluWj.mxB THlAdT6wffm.AtBCs.sbABFzdB2TQ47m6poZfJkQHfZ_Zv9XJIP5ogd_7FBfbV1xxzd85.xNFxxL rBVPb7LuNTwrHsfvla_L0vCugh.939vOyKwOdo6s2BXE3N1eln85NQRvOgwhtSZzvyBXnUc4mXw- - X-Sonic-MF: X-Sonic-ID: e869d467-c74b-4b72-b9c4-b3748f1e46cb Received: from sonic.gate.mail.ne1.yahoo.com by sonic303.consmr.mail.gq1.yahoo.com with HTTP; Thu, 6 Jul 2023 06:42:20 +0000 Received: by hermes--production-gq1-5748b5bccb-dgd7m (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 8ce3cf7c1303827ebf54c0af76c467ee; Thu, 06 Jul 2023 06:42:18 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.600.7\)) Subject: Re: For snapshot builds: armv7 chroot on aarch64 has kyua test -k /usr/tests/Kyuafile sys/kern/kern_copyin hung up [in getpid?], unkillable, prevents reboot Date: Wed, 5 Jul 2023 23:42:08 -0700 References: <7A41DED4-876F-4270-A980-549A4832B39A@yahoo.com> To: Current FreeBSD , freebsd-arm In-Reply-To: <7A41DED4-876F-4270-A980-549A4832B39A@yahoo.com> Message-Id: X-Mailer: Apple Mail (2.3731.600.7) X-Spamd-Result: default: False [-3.26 / 15.00]; NEURAL_HAM_LONG(-1.00)[-0.997]; NEURAL_HAM_MEDIUM(-0.99)[-0.993]; NEURAL_HAM_SHORT(-0.77)[-0.767]; MV_CASE(0.50)[]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; MIME_GOOD(-0.10)[text/plain]; FROM_HAS_DN(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; ARC_NA(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[98.137.64.204:from]; BLOCKLISTDE_FAIL(0.00)[98.137.64.204:server fail]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; MID_RHS_MATCH_FROM(0.00)[]; MLMMJ_DEST(0.00)[freebsd-arm@freebsd.org]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; RCVD_COUNT_THREE(0.00)[3]; FREEMAIL_FROM(0.00)[yahoo.com]; SUBJECT_HAS_QUESTION(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; RCVD_TLS_LAST(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; MIME_TRACE(0.00)[0:+]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.64.204:from] X-Rspamd-Queue-Id: 4QxRlx4hhvz4G2g X-Spamd-Bar: --- X-ThisMailContainsUnwantedMimeParts: N On Jun 25, 2023, at 17:16, Mark Millard wrote: > Using the likes of: >=20 > = FreeBSD-14.0-CURRENT-arm64-aarch64-ROCK64-20230622-b95d2237af40-263748.img= > and: > = FreeBSD-14.0-CURRENT-arm-armv7-GENERICSD-20230622-b95d2237af40-263748.img >=20 > I have shown the following behavior after setting up storage > media based on them. (This was a test that my builds were not > odd for the issue.) >=20 > Boot the aarch64 media and log in. (Note: I logged in > as root.) >=20 > mount the armv7 media (-noatime is just my habit) > and then put it to use: >=20 > # mount -onoatime /dev/da1s2a /mnt >=20 > # chroot /mnt/ >=20 > # kyua test -k /usr/tests/Kyuafile sys/kern/kern_copyin > sys/kern/kern_copyin:kern_copyin -> =20 >=20 > On the serial console: >=20 > # ps -xu > USER PID %CPU %MEM VSZ RSS TT STAT STARTED TIME COMMAND > root 11 1498.4 0.0 0 256 - RNL 23:24 542:52.92 [idle] > root 1174 100.0 0.0 0 16 - Rs 23:37 0:00.00 = /usr/tests/sys/kern/kern_copyin -vunprivileged-user=3Dtests = -r/tmp/kyua.9YUttj/2/result.atf kern_copyin > root 0 0.0 0.0 0 1616 - DLs 23:24 0:00.50 [kernel] > root 1 0.0 0.0 11704 1288 - ILs 23:24 0:00.02 /sbin/init > root 2 0.0 0.0 0 256 - WL 23:24 0:00.26 [clock] > root 3 0.0 0.0 0 272 - DL 23:24 0:00.00 [crypto] > root 4 0.0 0.0 0 80 - DL 23:24 0:00.95 [cam] > root 5 0.0 0.0 0 16 - DL 23:24 0:00.00 [busdma] > root 6 0.0 0.0 0 16 - DL 23:24 0:00.03 = [rand_harvestq] > root 7 0.0 0.0 0 48 - DL 23:24 0:00.06 = [pagedaemon] > root 8 0.0 0.0 0 16 - DL 23:24 0:00.00 [vmdaemon] > root 9 0.0 0.0 0 160 - DL 23:24 0:00.38 = [bufdaemon] > root 10 0.0 0.0 0 16 - DL 23:24 0:00.00 [audit] > root 12 0.0 0.0 0 880 - WL 23:24 0:11.81 [intr] > root 13 0.0 0.0 0 48 - DL 23:24 0:00.04 [geom] > root 14 0.0 0.0 0 16 - DL 23:24 0:00.00 [sequencer = 00] > root 15 0.0 0.0 0 160 - DL 23:24 0:06.42 [usb] > root 16 0.0 0.0 0 16 - DL 23:24 0:00.10 = [acpi_thermal] > root 17 0.0 0.0 0 16 - DL 23:24 0:00.00 = [acpi_cooling0] > root 18 0.0 0.0 0 16 - DL 23:24 0:00.04 [syncer] > root 19 0.0 0.0 0 16 - DL 23:24 0:00.00 [vnlru] > root 671 0.0 0.0 13260 2600 - Is 23:25 0:00.00 dhclient: = system.syslog (dhclient) > root 674 0.0 0.0 13260 2752 - Is 23:25 0:00.00 dhclient: = dpni0 [priv] (dhclient) > root 761 0.0 0.0 14572 3972 - Ss 23:25 0:00.02 /sbin/devd > root 964 0.0 0.0 12832 2764 - Is 23:25 0:00.02 = /usr/sbin/syslogd -s > root 1033 0.0 0.0 13012 2604 - Ss 23:25 0:00.01 = /usr/sbin/cron -s > root 1058 0.0 0.0 21052 8308 - Is 23:25 0:00.01 sshd: = /usr/sbin/sshd [listener] 0 of 10-100 startups (sshd) > root 1078 0.0 0.0 21288 9304 - Is 23:26 0:00.09 sshd: = root@pts/0 (sshd) > root 1175 0.0 0.0 21288 9496 - Is 23:37 0:00.04 sshd: = root@pts/1 (sshd) > root 1074 0.0 0.0 13380 3008 u0 Is 23:25 0:00.01 login = [pam] (login) > root 1075 0.0 0.0 13460 3292 u0 S 23:25 0:00.02 -sh (sh) > root 1233 0.0 0.0 13588 3016 u0 R+ 00:00 0:00.00 ps -xu > root 1081 0.0 0.0 13460 3328 0 Is 23:26 0:00.02 -sh (sh) > root 1170 0.0 0.0 5788 2884 0 I 23:36 0:00.02 /bin/sh -i > root 1172 0.0 0.0 10408 7192 0 I+ 23:37 0:00.30 kyua test = -k /usr/tests/Kyuafile sys/kern/kern_copyin > root 1178 0.0 0.0 13460 3320 1 Is+ 23:38 0:00.01 -sh (sh) >=20 > 1174 is stuck, even if one waits for 30min+. > kill and kill -9 will not kill 1174. >=20 > "shutdown -r now" hangs before the reboot happens > and reports: "some processes would not die". >=20 > An interesting property is that ps and top disagree > about 1174 CPU usage: ps 100%, top 0%. But top also > indicates 1174 always has CPU0 "STATE". (Across > tests CPUn varies but within a test it has > a fixed n.) >=20 > I have also seen ps "STAT" being RXs. >=20 > The following is from my earlier activity with my own > builds involved, here 1119, not the 1174 from above. > truss reports as the last thing for the stuck process > as "getpid()". >=20 > . . . > 1119: 0.588983953 fstatat(AT_FDCWD,"/usr/tests/sys/kern/kern_copyin",{ = mode=3D-r-xr-xr-x ,inode=3D111756,size=3D9776,blksize=3D10240 = },AT_SYMLINK_NOFOLLOW) =3D 0 (0x0) > 1119: 0.589065030 = mmap(0x0,20480,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANON|MAP_ALIGNED(12),-= 1,0x0) =3D 1074188288 (0x4006d000) > 1119: 0.589227544 = openat(AT_FDCWD,"/tmp/kyua.aBQv6E/2/result.atf",O_WRONLY|O_CREAT|O_TRUNC,0= 644) =3D 3 (0x3) > 1119: 0.589276503 getpid() =3D 1119 (0x45f) >=20 >=20 >=20 > For reference, from inside an armv7 chroot session > before doing such a test: >=20 > # uname -apKU > FreeBSD generic 14.0-CURRENT FreeBSD 14.0-CURRENT #0 = main-n263748-b95d2237af40: Thu Jun 22 11:10:50 UTC 2023 = root@releng1.nyi.freebsd.org:/usr/obj/usr/src/arm64.aarch64/sys/GENERIC = arm armv7 1400090 1400090 I've replicated the same sort of hangup based on: aarch64 (booted): # uname -apKU FreeBSD CA72-16Gp-ZFS 14.0-CURRENT FreeBSD 14.0-CURRENT #0 = n263893-0631830a7a3c-dirty: Wed Jul 5 13:54:15 PDT 2023 = root@CA72-16Gp-ZFS:/usr/obj/BUILDs/alt-main-CA72-nodbg-clang-alt/usr/alt-m= ain-src/arm64.aarch64/sys/GENERIC-NODBG-CA72 arm64 aarch64 1400092 = 1400092 armv7 (as seen in a chroot use): # uname -apKU FreeBSD CA72-16Gp-ZFS 14.0-CURRENT FreeBSD 14.0-CURRENT #0 = n263893-0631830a7a3c-dirty: Wed Jul 5 13:54:15 PDT 2023 = root@CA72-16Gp-ZFS:/usr/obj/BUILDs/alt-main-CA72-nodbg-clang-alt/usr/alt-m= ain-src/arm64.aarch64/sys/GENERIC-NODBG-CA72 arm armv7 1400092 1400092 =3D=3D=3D Mark Millard marklmi at yahoo.com