From nobody Tue Feb 14 19:29:28 2023 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PGWVf4hDcz3psxg for ; Tue, 14 Feb 2023 19:29:30 +0000 (UTC) (envelope-from jmg@gold.funkthat.com) Received: from gold.funkthat.com (gold.funkthat.com [IPv6:2001:470:800b::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "gate2.funkthat.com", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4PGWVf0xW5z3sP0; Tue, 14 Feb 2023 19:29:30 +0000 (UTC) (envelope-from jmg@gold.funkthat.com) Authentication-Results: mx1.freebsd.org; dkim=none; spf=none (mx1.freebsd.org: domain of jmg@gold.funkthat.com has no SPF policy when checking 2001:470:800b::2) smtp.mailfrom=jmg@gold.funkthat.com; dmarc=none Received: from gold.funkthat.com (localhost [127.0.0.1]) by gold.funkthat.com (8.15.2/8.15.2) with ESMTPS id 31EJTSL8023423 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Tue, 14 Feb 2023 11:29:28 -0800 (PST) (envelope-from jmg@gold.funkthat.com) Received: (from jmg@localhost) by gold.funkthat.com (8.15.2/8.15.2/Submit) id 31EJTSX9023422; Tue, 14 Feb 2023 11:29:28 -0800 (PST) (envelope-from jmg) Date: Tue, 14 Feb 2023 11:29:28 -0800 From: John-Mark Gurney To: Mark Millard , bob prohaska , "mckusick@freebsd.org" , freebsd-arm@freebsd.org Subject: Re: fsck segfaults on rpi3 running 13-stable (and on 14-CURRENT analyzing the same file system that resulted from the 13-STABLE crash) Message-ID: <20230214192928.GH95670@funkthat.com> Mail-Followup-To: Mark Millard , bob prohaska , "mckusick@freebsd.org" , freebsd-arm@freebsd.org References: <9DC74DD9-9AA1-4822-B425-217AAC7DB3F5@yahoo.com> <20230212043524.GA19401@www.zefox.net> <984314A1-FF42-4F92-A212-6BC0D85CB630@yahoo.com> <20230212165333.GB19401@www.zefox.net> <20230212191308.GA21535@www.zefox.net> <20230212195324.GB21535@www.zefox.net> <03840D0B-13D4-4F22-BDAF-2887A4D78BED@yahoo.com> <20230213232519.GD95670@funkthat.com> List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20230213232519.GD95670@funkthat.com> X-Operating-System: FreeBSD 11.3-STABLE amd64 X-PGP-Fingerprint: D87A 235F FB71 1F3F 55B7 ED9B D5FF 5A51 C0AC 3D65 X-Files: The truth is out there X-URL: https://www.funkthat.com/ X-Resume: https://www.funkthat.com/~jmg/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? User-Agent: Mutt/1.6.1 (2016-04-27) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (gold.funkthat.com [127.0.0.1]); Tue, 14 Feb 2023 11:29:29 -0800 (PST) X-Spamd-Result: default: False [-1.79 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; AUTH_NA(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.99)[-0.989]; FORGED_SENDER(0.30)[jmg@funkthat.com,jmg@gold.funkthat.com]; MIME_GOOD(-0.10)[text/plain]; R_SPF_NA(0.00)[no SPF record]; ASN(0.00)[asn:6939, ipnet:2001:470::/32, country:US]; MLMMJ_DEST(0.00)[freebsd-arm@freebsd.org]; FREEMAIL_TO(0.00)[yahoo.com,www.zefox.net,FreeBSD.org,freebsd.org]; MIME_TRACE(0.00)[0:+]; R_DKIM_NA(0.00)[]; TO_DN_EQ_ADDR_SOME(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; FROM_NEQ_ENVFROM(0.00)[jmg@funkthat.com,jmg@gold.funkthat.com]; FROM_HAS_DN(0.00)[]; FREEFALL_USER(0.00)[jmg]; ARC_NA(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; RCVD_TLS_LAST(0.00)[]; TO_DN_SOME(0.00)[]; DMARC_NA(0.00)[funkthat.com]; MID_RHS_MATCH_FROM(0.00)[] X-Rspamd-Queue-Id: 4PGWVf0xW5z3sP0 X-Spamd-Bar: - X-ThisMailContainsUnwantedMimeParts: N John-Mark Gurney wrote this message on Mon, Feb 13, 2023 at 15:25 -0800: After analyzing the remaining crashes that were detected, the following was found.. This was done w/ a minimal FS, It is possible that it'll find more crashes w/ a slightly more complex FS, e.g. one w/ indirect blocks, and other file types. Note that FS's that exhibit these are available at: https://www.funkthat.com/~jmg/FreeBSD/ffs.afl/ ---- FIRST ---- std_checkblkavail doesn't check that blkno is within valid range, e.g. if blkno is negative, it'll segfault: 0x0000000000121d6c in std_checkblkavail (blkno=blkno@entry=-288230376151711688, frags=frags@entry=1) at /usr/src/sbin/fsck_ffs/fsutil.c:1149 1149 if (testbmap(blkno + j)) ---- SECOND ---- pass5 doesn't check that cg_nextfreeoff is sane/within range, this causes a large value to be passed to memset, in this case: #1 0x000000000012fee4 in pass5 () at /usr/src/sbin/fsck_ffs/pass5.c:241 241 memset(cg_inosused(newcg), 0, (size_t)(mapsize)); (gdb) print *newcg $7 = {cg_firstfield = 0, cg_magic = 590421, cg_old_time = 0, cg_cgx = 0, cg_old_ncyl = 0, cg_old_niblk = 0, cg_ndblk = 256, cg_cs = {cs_ndir = 0, cs_nbfree = 0, cs_nifree = 128, cs_nffree = 0}, cg_rotor = 0, cg_frotor = 0, cg_irotor = 0, cg_frsum = {0, 0, 0, 0, 0, 0, 0, 0}, cg_old_btotoff = 0, cg_old_boff = 0, cg_iusedoff = 168, cg_freeoff = 184, cg_nextfreeoff = 61341980, cg_clustersumoff = 54526164, cg_clusteroff = 54526232, cg_nclusterblks = 32, cg_niblk = 128, cg_initediblk = 128, cg_unrefs = 0, cg_sparecon32 = {0}, cg_ckhash = 3548071837, cg_time = 1676327229, cg_sparecon64 = {0, 0, 0}, cg_space = ""} and mapsize is: (gdb) print newcg->cg_nextfreeoff - newcg->cg_iusedoff $6 = 61341812 which overflows buf, which is MAXBSIZE, or 8k. ---- THIRD ---- allocino doesn't make sure that cg_iusedoff is sane. In this case, cg_iusedoff is 4294965672, which means that in allocino, the setbit function call will access invalid memory. 0x0000000000127168 in allocino (request=2, type=16877) at /usr/src/sbin/fsck_ffs/inode.c:1379 1379 setbit(cg_inosused(cgp), ino % sblock.fs_ipg); (gdb) print *cgp $14 = {cg_firstfield = 0, cg_magic = 590421, cg_old_time = 0, cg_cgx = 0, cg_old_ncyl = 0, cg_old_niblk = 0, cg_ndblk = 256, cg_cs = {cs_ndir = 2, cs_nbfree = 23, cs_nifree = 124, cs_nffree = 21}, cg_rotor = 0, cg_frotor = 0, cg_irotor = 0, cg_frsum = {0, 0, 0, 0, 0, 0, 0, 3}, cg_old_btotoff = 0, cg_old_boff = 0, cg_iusedoff = 4294965672, cg_freeoff = 183, cg_nextfreeoff = 284, cg_clustersumoff = 212, cg_clusteroff = 280, cg_nclusterblks = 32, cg_niblk = 128, cg_initediblk = 128, cg_unrefs = 0, cg_sparecon32 = {0}, cg_ckhash = 3548071837, cg_time = 1676327229, cg_sparecon64 = {0, 0, 0}, cg_space = "\017"} -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not."