Re: Unusual errors on recent stable/13 22-Dec-2022 (a related problem report on freebsd-ports?)
Date: Fri, 23 Dec 2022 23:47:33 UTC
Jonathan Chen <jonc_at_chen.org.nz> wrote on Date: Fri, 23 Dec 2022 18:40:27 UTC : > On 24/12/22 07:14, Mark Millard wrote: > > Jonathan Chen <jonc_at_chen.org.nz> wrote on > > Date: Thu, 22 Dec 2022 19:21:37 UTC : > > > >> I recently updated my package builder machine to the > >> stable/13-n253297-fc15d5bf1109of (22-Dec-2022); and appear to be having > >> some unusual issues when building with a high number of jobs. My package > >> builder uses synth (which uses nullfs on ZFS), and I have had failures > >> with missing files, as well as what appears to be sequencing issues with > >> Makefiles. If I re-run the build, these errors usually do not reoccur. > >> > >> I'm puzzled as to what is happening. Is this just happening to me? It > >> appears that the ZFS code has been updated recently, and I'm wondering > >> whether a regression has crept in. [Or it could just be my hardware?] > > > > > > https://lists.freebsd.org/archives/freebsd-ports/2022-December/003153.html > > > > indicates a problem with tmpfs use such that using USE_TMPFS=no > > avoids a problem for poudriere bulk builds on 13.1 amd64. > > (Unclear if the note is for stable/13 vs. releng/13.1 vs. both.) > > I'll try disabling tmpfs on synth. > FYI . . . The following is about the tmpfs issue referenced in freebsd-ports/2022-December/003153.html . Here is what is going on (manually entered commands, not a script). First under a tmpfs: # df -m Filesystem 1M-blocks Used Avail Capacity Mounted on /dev/ufs/rootfs 221683 97879 106068 48% / devfs 0 0 0 100% /dev /dev/msdosfs/MSDOSBOOT 49 31 18 62% /boot/msdos tmpfs 7716 0 7716 0% /tmp # cd /tmp # : > mmjnk.test # ls -Tld mmjnk.test -rw-r--r-- 1 root wheel 0 Mar 9 08:56:53 2022 mmjnk.test # : > mmjnk.test # ls -Tld mmjnk.test -rw-r--r-- 1 root wheel 0 Mar 9 08:56:53 2022 mmjnk.test (no time change). The makefile involved is using ": > NAME" notation to try to update timestamps on deliberately empty files. Vs. under (for example) UFS: # cd ~/ # : > mmjnk.test # ls -Tld mmjnk.test -rw-r--r-- 1 root wheel 0 Mar 9 09:00:45 2022 mmjnk.test # : > mmjnk.test # ls -Tld mmjnk.test -rw-r--r-- 1 root wheel 0 Mar 9 09:00:54 2022 mmjnk.test (time changed). Back in tmpfs land . . . Part of this is that the file is already of size zero and continues to be so. By contrast, starting with a file with 15 bytes in it: # ls -Tld mmjnk.test -rw-r--r-- 1 root wheel 15 Mar 9 09:07:38 2022 mmjnk.test # : > mmjnk.test # ls -Tld mmjnk.test -rw-r--r-- 1 root wheel 0 Mar 9 09:07:49 2022 mmjnk.test # ls -Tld mmjnk.test -rw-r--r-- 1 root wheel 0 Mar 9 09:07:49 2022 mmjnk.test The lack of a timestamp change when the file already has size zero looks like an example of a bug to me. truncate for tmpfs files behaves similarly (showing just the lack of timestamp change context): # truncate -s 0 mmjnk.test # ls -Tld mmjnk.test -rw-r--r-- 1 root wheel 0 Mar 9 09:11:31 2022 mmjnk.test # truncate -s 0 mmjnk.test # ls -Tld mmjnk.test -rw-r--r-- 1 root wheel 0 Mar 9 09:11:31 2022 mmjnk.test (UFS got a timestamp update from such a sequence.) I'll note that touch does not get this tmpfs behavior: # touch mmjnk.test # ls -Tld mmjnk.test -rw-r--r-- 1 root wheel 0 Mar 9 09:11:26 2022 mmjnk.test # touch mmjnk.test # ls -Tld mmjnk.test -rw-r--r-- 1 root wheel 0 Mar 9 09:11:31 2022 mmjnk.test (But it would not force size zero on its down.) I did these tests on: # uname -apKU FreeBSD generic 13.1-STABLE FreeBSD 13.1-STABLE #0 stable/13-n253133-b51ee7ac252c: Wed Nov 23 03:36:16 UTC 2022 root@releng3.nyi.freebsd.org:/usr/obj/usr/src/arm64.aarch64/sys/GENERIC arm64 aarch64 1301509 1301509 However, I previously did a devel/nasm bulk test with with USE_TMPFS=all on: # uname -apKU FreeBSD amd64_ZFS 14.0-CURRENT FreeBSD 14.0-CURRENT #55 main-n259064-f83db6441a2f-dirty: Sun Nov 6 16:31:55 PST 2022 root@amd64_ZFS:/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/sys/GENERIC-NODBG amd64 amd64 1400073 1400073 and it got the problem. (I normally use USE_TMPFS=data , which does not get the problem because the files in question end up not on a tmpfs.) So: not specific to amd64 , not specific to stable/13 , existed in early November in main. This may have been around for some time for tmpfs. === Mark Millard marklmi at yahoo.com