From nobody Wed Oct 18 10:10:27 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4S9RTF3qcgz4xpZ8 for ; Wed, 18 Oct 2023 10:11:29 +0000 (UTC) (envelope-from Alexander@Leidinger.net) Received: from mailgate.Leidinger.net (mailgate.leidinger.net [IPv6:2a00:1828:2000:313::1:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature ECDSA (P-256) client-digest SHA256) (Client CN "mailgate.leidinger.net", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4S9RTD4NnLz4FtM for ; Wed, 18 Oct 2023 10:11:28 +0000 (UTC) (envelope-from Alexander@Leidinger.net) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=leidinger.net header.s=outgoing-alex header.b=RHvYNyHp; spf=pass (mx1.freebsd.org: domain of Alexander@Leidinger.net designates 2a00:1828:2000:313::1:5 as permitted sender) smtp.mailfrom=Alexander@Leidinger.net; dmarc=pass (policy=quarantine) header.from=leidinger.net List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=leidinger.net; s=outgoing-alex; t=1697623874; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=ZRtmh9oK6Dc5s/UFAxYtsXrix5EvNq8ou1R9RLuFNbo=; b=RHvYNyHpl5g1Y4D1GuOC07nm73S1rczSb7tUaxcI3IFiEpBJJUAtzTyXMuNV945S6UK/ew /gDf2wrH5dcIC5YKynR1A08X+UjrWX+yiMshUDflXNBDxuVyzJKn43LcjhPq/ZyNlQXXC3 Vohpa4s76Bx00pzKJAli+z+bS1yvUOxmLnq4Sho/bHt2fhKXRDkqksPmPA8kuldLqYoLbO qgoxrGBSu6CGNpzyPYVIPCPHbw5AZJT8+Xw8hktRlk3o5a37wEyMUV7h1CHilpVoeg31uV yVEgpar9LygBpkoqKaKEWLbh7qr2Zx89ukY87CSs+9KAgEel6jOCiv2bR8Bb6g== Date: Wed, 18 Oct 2023 12:10:27 +0200 From: Alexander Leidinger To: freebsd-current@freebsd.org Subject: Re: poudriere job && find jobs which received signal 11 In-Reply-To: References: Message-ID: X-Sender: Alexander@Leidinger.net Organization: No organization, this is a private message. Content-Type: multipart/signed; protocol="application/pgp-signature"; boundary="=_534d6f9491070f4ee72b1465ac426e56"; micalg=pgp-sha256 X-Spamd-Bar: ------ X-Spamd-Result: default: False [-6.10 / 15.00]; SIGNED_PGP(-2.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.999]; DMARC_POLICY_ALLOW(-0.50)[leidinger.net,quarantine]; R_SPF_ALLOW(-0.20)[+mx]; MIME_GOOD(-0.20)[multipart/signed,text/plain]; R_DKIM_ALLOW(-0.20)[leidinger.net:s=outgoing-alex]; RCVD_COUNT_ZERO(0.00)[0]; FROM_EQ_ENVFROM(0.00)[]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ARC_NA(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; HAS_ORG_HEADER(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; DKIM_TRACE(0.00)[leidinger.net:+]; HAS_ATTACHMENT(0.00)[]; ASN(0.00)[asn:34240, ipnet:2a00:1828::/32, country:DE]; TO_DN_NONE(0.00)[]; MID_RHS_MATCH_FROM(0.00)[] X-Rspamd-Queue-Id: 4S9RTD4NnLz4FtM This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --=_534d6f9491070f4ee72b1465ac426e56 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII; format=flowed Am 2023-10-18 09:54, schrieb Matthias Apitz: > Hello, > > I'm compiling with poudriere on 14.0-CURRENT 1400094 amd64 "my" ports, > from git October 14, 2023. In the last two day 2229 packages were > produced fine, on job failed (p5-Gtk2-1.24993_3 for been known broken). > > This morning I was looking for something in /var/log/messages and > accidentally I detected that yesterday a few compilations failed: > > # grep 'signal 11' /var/log/messages | grep -v conftest > Oct 17 10:58:02 jet kernel: pid 12765 (cc1plus), jid 24, uid 65534: > exited on signal 11 (core dumped) > Oct 17 10:59:32 jet kernel: pid 27104 (cc1plus), jid 24, uid 65534: > exited on signal 11 (core dumped) > Oct 17 12:07:38 jet kernel: pid 85640 (cc1plus), jid 24, uid 65534: > exited on signal 11 (core dumped) > Oct 17 12:08:17 jet kernel: pid 94451 (cc1plus), jid 24, uid 65534: > exited on signal 11 (core dumped) > Oct 17 12:36:01 jet kernel: pid 77914 (cc1plus), jid 24, uid 65534: > exited on signal 11 (core dumped) > > As I said, without that any of the 2229 jobs were failing: > > # cd > /usr/local/poudriere/data/logs/bulk/140-CURRENT-ports20231014/latest-per-pkg > # ls -C1 | wc -l > 2229 > # grep -l 'build failure' * > p5-Gtk2-1.24993_3.log > > How this is possible, that the make engines didn't failing? The uid That can be part of configure runs which try to test some features. > 65534 is the one used by poudriere, can I use the jid 24 somehow to > find > the job which received the signal 11? Or is the time the only way to jid = jail ID, the first column in the output of "jls". If you have the poudriere runtime logs (where it lists which package it is processing ATM), you will see a number from 1 to the max number of jails which run in parallel. This number is part of the hostname of the jail. So if you have the poudriere jails still running, you can make a mapping from the jid to the name to the number, and together with the time you can see which package it was building at that time. Unfortunately poudriere doesn't list the hostname of the builder nor the jid (feature request anyone?). Example poudriere runtime log: ---snip--- [00:54:11] [03] [00:00:00] Building security/nss | nss-3.94 [00:56:46] [03] [00:02:35] Finished security/nss | nss-3.94: Success [00:56:47] [03] [00:00:00] Building textproc/gsed | gsed-4.9 [00:57:41] [01] [00:06:18] Finished x11-toolkits/gtk30 | gtk3-3.24.34_1: Success [00:57:42] [01] [00:00:00] Building devel/qt6-base | qt6-base-6.5.3 ---snip--- While poudriere is running, jls reports this: ---snip--- # jls jid host.hostname [...] 91 poudriere-bastille-default 92 poudriere-bastille-default 93 poudriere-bastille-default-job-01 94 poudriere-bastille-default-job-01 95 poudriere-bastille-default-job-02 96 poudriere-bastille-default-job-03 97 poudriere-bastille-default-job-02 98 poudriere-bastille-default-job-03 ---snip--- So if we assume a coredump in jid 96 or 98, this means it was in builder 3. nss and gseed where build by poudriere builder number 3 (both about 56 minutes after start of poudriere), and gtk30 and qt6-base by poudriere builder number 1. If we assume further that the coredumps are in the timerange of 54 to 56 minutes after the poudriere start, the logs of nss may have a trace of it (or not, if it was part of configure, then you would have to do the configure run and check the messages if it generates similar coredumps) > look, which of the 4 poudriere engines were running at this time? > I'd like to rerun/reproduce the package again. Bye, Alexander. -- http://www.Leidinger.net Alexander@Leidinger.net: PGP 0x8F31830F9F2772BF http://www.FreeBSD.org netchild@FreeBSD.org : PGP 0x8F31830F9F2772BF --=_534d6f9491070f4ee72b1465ac426e56 Content-Type: application/pgp-signature; name=signature.asc Content-Disposition: attachment; filename=signature.asc; size=833 Content-Description: OpenPGP digital signature -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEER9UlYXp1PSd08nWXEg2wmwP42IYFAmUvryEACgkQEg2wmwP4 2IaZnRAAi0Z7nFlN5IKN3nykQ+KiEmMS+3HusKf7ksK/LTPirY/gBpmeC1qRMaS0 aXtYewyqfsiFbHXXnufZt2PJp4SNjTqEWD5yiQs0ZvJ2ZQoScmU76wkkbRG9BeGC 9UV4jDQRTH9kQNwPRj6RxL2oJZ650xJZwHYCU8DKNgfbpibnis0UKZI55V18f+6t QhQrcGBc7rUmFCkwsGeMb5QcsC7BovRXrxM5J2Mh4w9bKCKwwv8xl9TIYoHQqRPJ R2pcRKtgc32je6H2RdG6MnMdp9CXAjPD0tll7HOMF/lncZIjsfkRPgLUzEtybrEA 35CsTnkWHwb9jkHy/Og3WYmah4fCZmek6NiuAs4BtDZGZZk/Ur5g1QI7r52moy1V 5Jg8aQlCuCGCg0dxXkyAIDgFqUdaT6i1L97izo18PnaK4OfdT/mW8co7x0Ex2nkw dxaCId2zYlg8jEB0udSyIQmjLTmnIZXxD5eP5lrvCCj7843OXN8MnnRQJKPtCgU7 2H2mLaGXIZOxf0PPrBf9WjfjiZz0Q4D3QgrfzO9jnChwYSipzxK43VeqlD05WxOU IQva/fjflUrrTpl6WVg2qQMUMK5iqdnhv0QNIFxxip3DYU66RHiRAYWyDcDiENfQ AzOkwI0bO5ipEj5/rBjxqL9sFmZBmErJlR8SRi58BiwnaqkrvDs= =v/5A -----END PGP SIGNATURE----- --=_534d6f9491070f4ee72b1465ac426e56--