From nobody Tue Nov 26 17:29:23 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4XyV1g4DJmz5fkcx; Tue, 26 Nov 2024 17:29:27 +0000 (UTC) (envelope-from unkadoug@gmail.com) Received: from mail-oi1-x22d.google.com (mail-oi1-x22d.google.com [IPv6:2607:f8b0:4864:20::22d]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4XyV1g2KTGz4x4Q; Tue, 26 Nov 2024 17:29:27 +0000 (UTC) (envelope-from unkadoug@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-oi1-x22d.google.com with SMTP id 5614622812f47-3ea696c4dcaso299027b6e.2; Tue, 26 Nov 2024 09:29:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1732642165; x=1733246965; darn=freebsd.org; h=in-reply-to:references:cc:to:content-language:subject:user-agent :mime-version:date:message-id:from:from:to:cc:subject:date :message-id:reply-to; bh=jhbT+hQoxcEr8V7GXf6nJnWgsO+6vE+8Vte1YuFS8C0=; b=ZDu+rntCMhaR/cAqZLBzMuBmbVHkcJjdaPpwjWs+3a8IVVHK86LW505+j6Ud88QYKv bLil4mOKYHWrJO/C85vZDRBzQQiFVP2Ah+C8GOTJ+USpjC+Vld3UkEmbrf82EuryxAHa 05YwFlfqPKb9vB//xjIi9R3I7NABEAX/eAWMYpNHT1mfj2GYw64pnVRXM8xbqQF54giQ LhoZ+RJkuPZs0tb+Nogil2hNlQPy3k0UVv+dAxdMIR8tnWckdsQ3zgKYI1JZmMWEvCG+ Q4xtgMLdktpQ+mpq8ysd1NQgvbV5EmwiMmSL3vP7vVYi2qxESyiN0fZjW4FfFFaT8eY5 qbbQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1732642165; x=1733246965; h=in-reply-to:references:cc:to:content-language:subject:user-agent :mime-version:date:message-id:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=jhbT+hQoxcEr8V7GXf6nJnWgsO+6vE+8Vte1YuFS8C0=; b=cd/niDHXCQnIW/4qQ5GaR37X39ej988tweMp/T/R7FJuTWTZAB8f72dE5r1PMEvLrz VI2y3l1NAnNDQx8xEuvP6UK6he25a99WH0HVzLQjCR9pnkAvEqXDIu7CB/lVc4Icb+Us 82TfLxZkvT3kVP8392CkFrdFwLkto9KzU7jnhOz8wdDnHDWMeTkvVCtPtZwbtrJzhAPB QvtSErHsqrGI8I9OqTK7h6m6rCEf9qjp7LOhLGLCUu6qswnnuikV8VOfqpAhPzf+IfZk HQrtZp41kwemZcmCBSqG8PpXX0XodlgaIsEbYqi8WiTp7EdOtKqS6VVO/iJBuXVBC1Ga OW4g== X-Forwarded-Encrypted: i=1; AJvYcCU9c7kXYN8iZnkGIqoKueG4RFYfnXYikRPrbq8ND9AJRNRGfCNwmdRuQYFFeMeybwkpkqog@freebsd.org, AJvYcCUN/HTPxnoLkroh5Z9CY6LyL7GeFOdtDIK75MHLePsJWdcnQa7sWZ3VLB05vHt9US6RSG7cmQk=@freebsd.org, AJvYcCUlec/zvCon3WiQiugOW63DPUe7nFQ122jnElZyYt0vX3uxrTqn8C8Moh4z/LbxICkAScMKW1j16C7VCc3pC65R@freebsd.org, AJvYcCUqRRDuA1J3tjZ6Eop2ukPyoaGnnEDVV0w4QfjsQR+4kwypG56r1tO7HkgQ+2WCqYabDxWEGA==@freebsd.org, AJvYcCWbzCO5hCXZ13nAr3nkIZ6hvphyz7t+qQToS4sNdzrpde8YtwaWoIiKjgf/D2Dz67XPndaqqLc=@freebsd.org, AJvYcCWvalVXjuPQ0tIHUQRa7r2s7xMu/DxGZCUeZvj3UOIdzlMsTnI+bDdTTpw2IC5kFGVW3lJNlRY/@freebsd.org, AJvYcCXetwrs+Ru9JxOxb4mpqYhcDbAWc0V/0AvzxWD4sli0P0X5db+j5XEst+YSmnySpfZz4HuN@freebsd.org, AJvYcCXjb1INcBcs82qQeqdcKOI9He9q7wlMr2A29qDZcAUwXQKG4/H2J4hSiDadtcIWeeYk+UoR@freebsd.org X-Gm-Message-State: AOJu0YxvPwvPYeH6In0Fp+1hEC0HWahJwx0AO9czwXt04CzuG1nGOEqO sUSX4ioY1636N4kzymVqWiw6TY+nUbT2qsJKZrbJfze+SX/g54TGKHeOs6CM X-Gm-Gg: ASbGnctpes4snoFK7QwUcw8/p2yyPIQorkty3qYBUSbQsmTBggnCPv6Ji9Ajpr28kj2 wfq7PIMtgA4VroF7Xf1o5o0u8vRl7Nmd3mC2smKdfei5seGEReeL0WQU5xuyXn/4JFufkaS8xnp gU/f43nT+dzwD956WDMUZ6JtPTa6oXCcZJM4+jbo35SweyksIRwymG+gbYn4ufIXL1tYYr+jTjn g9xFITNABZNWqZ1Lus6Zazk0NfOK+MceIDyh207Z1WKnFDntlsYpf1oxh3AAy0kMnqgalNTfEqp VnJbXI+nBg3h6MuW+yAOKqWY0T9+3Sg3 X-Google-Smtp-Source: AGHT+IH4dj8GoAtsSCgESZPL6BULKHlafkDX8sNE/mZWmccDBf2m0hDyJFg9LjvRGg7bq7lq3FYqrg== X-Received: by 2002:a05:6808:2025:b0:3ea:4140:e7dd with SMTP id 5614622812f47-3ea6dd48c3emr128702b6e.32.1732642165445; Tue, 26 Nov 2024 09:29:25 -0800 (PST) Received: from [108.254.203.202] (108-254-203-202.lightspeed.hstntx.sbcglobal.net. [108.254.203.202]) by smtp.gmail.com with ESMTPSA id 5614622812f47-3ea542d4033sm843593b6e.35.2024.11.26.09.29.24 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 26 Nov 2024 09:29:24 -0800 (PST) From: Doug Moore X-Google-Original-From: Doug Moore Content-Type: multipart/mixed; boundary="------------dLvnPAVEPNGf2FLWnmem0aRA" Message-ID: Date: Tue, 26 Nov 2024 11:29:23 -0600 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:102.0) Gecko/20100101 Thunderbird/102.10.0 Subject: Re: port binary dumping core on recent head in poudriere [tmpfs corruptions involving blocks of zeros that should not be all zeros] Content-Language: en-US To: Mark Millard , Konstantin Belousov Cc: Dimitry Andric , =?UTF-8?Q?Dag-Erling_Sm=c3=b8rgrav?= , "jah@freebsd.org" , Alan Somers , Mark Johnston , FreeBSD Current , Guido Falsi , Yasuhiro Kimura , ports@freebsd.org References: <38658C0D-CA33-4010-BBE1-E68D253A3DF7@FreeBSD.org> <1004a753-9a3c-4aa2-bfa8-4a0c471fe3ea@madpilot.net> <0690CFB1-6A6D-4B63-916C-BAB7F6256000@yahoo.com> <3660625A-0EE8-40DA-A248-EC18C734718C@yahoo.com> <865xoa2t6f.fsf@ltc.des.dev> <69A2E921-F5E3-40D2-977D-0964EE27349A@FreeBSD.org> <4AE5B316-D7EB-4290-8D52-7FBF244EA7A4@FreeBSD.org> <33D56E3E-6476-48E8-B115-B906629B8AF5@yahoo.com> In-Reply-To: <33D56E3E-6476-48E8-B115-B906629B8AF5@yahoo.com> X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] X-Rspamd-Queue-Id: 4XyV1g2KTGz4x4Q X-Spamd-Bar: ---- This is a multi-part message in MIME format. --------------dLvnPAVEPNGf2FLWnmem0aRA Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit I think @kib has found the source of the problem.  I've attached an attempt to fix it. On 11/26/24 09:52, Mark Millard wrote: > On Nov 26, 2024, at 05:38, Konstantin Belousov wrote: > >> On Tue, Nov 26, 2024 at 01:58:19PM +0100, Dimitry Andric wrote: >>> On 26 Nov 2024, at 13:32, Dimitry Andric wrote: >>>> On 26 Nov 2024, at 11:19, Dag-Erling Smørgrav wrote: >>>>> Mark Millard writes: >>>>>> From inside a bulk -i where I did a manual make command >>>>>> after it built and installed libsass.so.1.0.0 . The >>>>>> manual make produced a /wrkdirs/ : >>>>>> [...] >>>>>> So the original creation looks okay. But . . . >>>>>> [...] >>>>>> So: The later, staged copy is a bad copy. Both are in the >>>>>> tmpfs. So copying to the staging area makes a corrupted >>>>>> copy inside the same tmpfs. After that, further copies of >>>>>> staging's bad copy can be expected to be messed up. >>>>> This and the fact that it happens on 14 and 15 but not on 13 strongly >>>>> suggests an issue wth `copy_file_range(2)`, since `install(1)` in 14 and >>>>> 15 (but not in 13) now uses `copy_file_range(2)` if at all possible. >>>>> >>>>> My educated guess is that hole detection doesn't work reliably for files >>>>> that have had holes filled while memory-mapped, so `copy_file_range(2)` >>>>> thinks there is a hole where there isn't one and skips some of the data >>>>> when `install(1)` uses it to copy the library from `${WRKSRC}` to >>>>> `${STAGEDIR}`. This may or may not be specific to tmpfs. >>>>> >>>>> You may want to try applying the attached patch to your FreeBSD 14 and >>>>> 15 jails. It prevents `cp(1)` and `install(1)` from trying to use >>>>> `copy_file_range(2)`. >>>> Yes, tmpfs is indeed the culprit (or at least involved). I have had USE_TMPFS=localbase in my poudriere.conf for a long time, since otherwise my build machine would run out of memory very quickly, so I didn't encounter any issues. >>>> >>>> Now I changed it to USE_TMPFS=yes, rebuilt only textproc/libsass and textproc/sassc, and then after reinstalling those packages: >>>> >>>> $ /usr/local/bin/sassc >>>> Segmentation fault >>> And after applying Dag-Erling's patch to disable copy_file_range for cp and install, it works correctly again. >> So indeed there might be an issue in tmpfs seeking for data. Could you try >> the following? >> >> commit f4b848946a131dab260b44eab2cfabceb82bee0c >> Author: Konstantin Belousov >> Date: Tue Nov 26 15:34:56 2024 +0200 >> >> tmpfs: do not skip pages searching for data >> >> If the iterator finds invalid page at the requested pindex in >> swap_pager_seek_data(), the current code only looks at the swap blocks >> to search for data. This is not correct, valid pages may appear at the >> higher indexes still. >> >> diff --git a/sys/vm/swap_pager.c b/sys/vm/swap_pager.c >> index db925f4ae7f6..390b2c10d680 100644 >> --- a/sys/vm/swap_pager.c >> +++ b/sys/vm/swap_pager.c >> @@ -2503,12 +2503,9 @@ swap_pager_seek_data(vm_object_t object, vm_pindex_t pindex) >> VM_OBJECT_ASSERT_RLOCKED(object); >> vm_page_iter_init(&pages, object); >> m = vm_page_iter_lookup_ge(&pages, pindex); >> - if (m != NULL) { >> - if (!vm_page_any_valid(m)) >> - m = NULL; >> - else if (pages.index == pindex) >> - return (pages.index); >> - } >> + if (m != NULL && pages.index == pindex) >> + return (pages.index); >> + >> swblk_iter_init_only(&blks, object); >> swap_index = swap_pager_iter_find_least(&blks, pindex); >> if (swap_index == pindex) > Not sufficient, unfortunately . . . > > I patched what I've been running and rebooted into: > > # uname -apKU > FreeBSD 7950X3D-ZFS 15.0-CURRENT FreeBSD 15.0-CURRENT #152 main-n273696-43e045c1733d-dirty: Tue Nov 26 07:21:27 PST 2024 root@7950X3D-ZFS:/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/sys/GENERIC-NODBG amd64 amd64 1500027 1500027 > > Note: 43e045c1733d is from 2024-Nov-18 . > > I then built libsass : > > [00:00:02] [01] [00:00:00] Building textproc/libsass | libsass-3.6.6 > [00:00:20] [01] [00:00:18] Finished textproc/libsass | libsass-3.6.6: Success ending TMPFS: 3.42 GiB > > I then installed it, resulting in: > > # pkg info libsass > libsass-3.6.6 > Name : libsass > Version : 3.6.6 > Installed on : Tue Nov 26 07:33:15 2024 PST > Origin : textproc/libsass > Architecture : FreeBSD:15:amd64 > Prefix : /usr/local > Categories : textproc > Licenses : MIT > Maintainer : nivit@FreeBSD.org > WWW : https://sass-lang.com/libsass > Comment : C/C++ implementation of a Sass compiler > Shared Libs provided: > libsass.so.1 > Annotations : > FreeBSD_version: 1500027 > build_timestamp: 2024-11-26T15:32:33+0000 > built_by : poudriere-git-3.4.99.20240811 > . . . > > libsass.so.1.0.0 still has .got.plt starting with (this time): > > 2bed60 00000000 00000000 00000000 00000000 ................ > 2bed70 00000000 00000000 00000000 00000000 ................ > 2bed80 00000000 00000000 00000000 00000000 ................ > 2bed90 00000000 00000000 00000000 00000000 ................ > . . . > 2bffc0 00000000 00000000 00000000 00000000 ................ > 2bffd0 00000000 00000000 00000000 00000000 ................ > 2bffe0 00000000 00000000 00000000 00000000 ................ > 2bfff0 00000000 00000000 00000000 00000000 ................ > 2c0000 96cb2a00 00000000 a6cb2a00 00000000 ..*.......*..... > 2c0010 b6cb2a00 00000000 c6cb2a00 00000000 ..*.......*..... > 2c0020 d6cb2a00 00000000 e6cb2a00 00000000 ..*.......*..... > 2c0030 f6cb2a00 00000000 06cc2a00 00000000 ..*.......*..... > . . . > > And still results in: > > # sassc > Segmentation fault (core dumped) > > > > === > Mark Millard > marklmi at yahoo.com > --------------dLvnPAVEPNGf2FLWnmem0aRA Content-Type: text/x-patch; charset=UTF-8; name="seek_data_fix.patch" Content-Disposition: attachment; filename="seek_data_fix.patch" Content-Transfer-Encoding: base64 ZGlmZiAtLWdpdCBhL3N5cy92bS9zd2FwX3BhZ2VyLmMgYi9zeXMvdm0vc3dhcF9wYWdlci5j CmluZGV4IGRiOTI1ZjRhZTdmNi4uM2QwMmYzNjVjYWQ5IDEwMDY0NAotLS0gYS9zeXMvdm0v c3dhcF9wYWdlci5jCisrKyBiL3N5cy92bS9zd2FwX3BhZ2VyLmMKQEAgLTI1MDMsMjYgKzI1 MDMsMjMgQEAgc3dhcF9wYWdlcl9zZWVrX2RhdGEodm1fb2JqZWN0X3Qgb2JqZWN0LCB2bV9w aW5kZXhfdCBwaW5kZXgpCiAJVk1fT0JKRUNUX0FTU0VSVF9STE9DS0VEKG9iamVjdCk7CiAJ dm1fcGFnZV9pdGVyX2luaXQoJnBhZ2VzLCBvYmplY3QpOwogCW0gPSB2bV9wYWdlX2l0ZXJf bG9va3VwX2dlKCZwYWdlcywgcGluZGV4KTsKLQlpZiAobSAhPSBOVUxMKSB7Ci0JCWlmICgh dm1fcGFnZV9hbnlfdmFsaWQobSkpCi0JCQltID0gTlVMTDsKLQkJZWxzZSBpZiAocGFnZXMu aW5kZXggPT0gcGluZGV4KQotCQkJcmV0dXJuIChwYWdlcy5pbmRleCk7Ci0JfQorCWlmICht ICE9IE5VTEwgJiYgcGFnZXMuaW5kZXggPT0gcGluZGV4ICYmIHZtX3BhZ2VfYW55X3ZhbGlk KG0pKQorCQlyZXR1cm4gKHBhZ2VzLmluZGV4KTsKIAlzd2Jsa19pdGVyX2luaXRfb25seSgm Ymxrcywgb2JqZWN0KTsKIAlzd2FwX2luZGV4ID0gc3dhcF9wYWdlcl9pdGVyX2ZpbmRfbGVh c3QoJmJsa3MsIHBpbmRleCk7CiAJaWYgKHN3YXBfaW5kZXggPT0gcGluZGV4KQogCQlyZXR1 cm4gKHN3YXBfaW5kZXgpOwotCWlmIChzd2FwX2luZGV4ID09IE9CSl9NQVhfU0laRSkKLQkJ c3dhcF9pbmRleCA9IG9iamVjdC0+c2l6ZTsKLQlpZiAobSA9PSBOVUxMKQotCQlyZXR1cm4g KHN3YXBfaW5kZXgpOwogCi0Jd2hpbGUgKChtID0gdm1fcmFkaXhfaXRlcl9zdGVwKCZwYWdl cykpICE9IE5VTEwgJiYKLQkgICAgcGFnZXMuaW5kZXggPCBzd2FwX2luZGV4KSB7CisJLyoK KwkgKiBGaW5kIHRoZSBmaXJzdCByZXNpZGVudCBwYWdlIGFmdGVyIG0sIGJlZm9yZSBzd2Fw X2luZGV4LgorCSAqLworCXdoaWxlIChtICE9IE5VTEwgJiYgcGFnZXMuaW5kZXggPCBzd2Fw X2luZGV4KSB7CiAJCWlmICh2bV9wYWdlX2FueV92YWxpZChtKSkKIAkJCXJldHVybiAocGFn ZXMuaW5kZXgpOworCQltID0gdm1fcmFkaXhfaXRlcl9zdGVwKCZwYWdlcyk7CiAJfQorCWlm IChzd2FwX2luZGV4ID09IE9CSl9NQVhfU0laRSkKKwkJc3dhcF9pbmRleCA9IG9iamVjdC0+ c2l6ZTsKIAlyZXR1cm4gKHN3YXBfaW5kZXgpOwogfQogCg== --------------dLvnPAVEPNGf2FLWnmem0aRA--