From nobody Sun Jan 28 21:34:34 2024 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TNPpX2HJsz59LHm; Sun, 28 Jan 2024 21:34:44 +0000 (UTC) (envelope-from mad@madpilot.net) Received: from mail.madpilot.net (vogon.madpilot.net [IPv6:2a01:4f8:1c1c:11e5::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4TNPpX0vmCz4LYh; Sun, 28 Jan 2024 21:34:44 +0000 (UTC) (envelope-from mad@madpilot.net) Authentication-Results: mx1.freebsd.org; none Received: from mail (mail [IPv6:fd5c:5351:d272::3]) by mail.madpilot.net (Postfix) with ESMTP id 4TNPpR0PjBz6g9X; Sun, 28 Jan 2024 22:34:39 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=madpilot.net; h= content-transfer-encoding:content-type:content-type:in-reply-to :from:from:references:content-language:subject:subject:date:date :message-id:received; s=bjowvop61wgh; t=1706477675; x= 1708292076; bh=5Cqr57rf3mXR1Ah5VgWPVeynOHP4RKyAAEqEesVgvHs=; b=p 93OMI3Ynz9j0QkJvl/ek+ZAwTFnbBM4bntfkaagPdfC/MfhEqI1YLrlCYR+EWnPX 4tVHU2Ybshp09BDkr4xYybyER8Mhy6Bd7o1/+YNPNqkUJiXKRtzKQRYrsZ4Q9227 ElsNywITZ7sT6R74c+3LcdqIkChVzU7nRXg8BMBqMwgiGPVhwQn8qlhZg/irRpOE 5nkTlLN0sJr5n5eHJhv0XnY9tPTnZPb8UU1LjSswocw+CfOQm0dklo0h3qEe3+Dc NHvqfCxw2DdFJ23taAU3lOZlpJvqo+K7LjE5zUw3+SrMWLxZAyWSjMUU8NggwLgS c9erE3OwH/MBtLxRk6CLA== Received: from mail.madpilot.net ([IPv6:fd5c:5351:d272::3]) by mail (mail.madpilot.net [IPv6:fd5c:5351:d272::3]) (amavisd-new, port 10026) with ESMTP id ScflYP6VeZzW; Sun, 28 Jan 2024 22:34:35 +0100 (CET) Message-ID: Date: Sun, 28 Jan 2024 22:34:34 +0100 Subject: Re: qemu-user-static aarch64 lockup/race? (was Re: Python failure in poudriere on arm64 (via qemu-user-static cross compiling)) Content-Language: en-US To: Warner Losh Cc: emulation@freebsd.org, "freebsd-arm@freebsd.org" References: <6a33726b-eb6f-418e-9fbd-6d0b9b4bfaa8@madpilot.net> <0fc7f929-6e5b-4a33-97d2-8a9c0c07d524@madpilot.net> <79a5eb0f-d04e-4c1a-9d8a-185e1fb4e4a2@madpilot.net> From: Guido Falsi Autocrypt: addr=mad@madpilot.net; keydata= xsBNBE+G+l0BCADi/WBQ0aRJfnE7LBPsM0G3m/m3Yx7OPu4iYFvS84xawmRHtCNjWIntsxuX fptkmEo3Rsw816WUrek8dxoUAYdHd+EcpBcnnDzfDH5LW/TZ4gbrFezrHPdRp7wdxi23GN80 qPwHEwXuF0X4Wy5V0OO8B6VT/nA0ADYnBDhXS52HGIJ/GCUjgqJn+phDTdCFLvrSFdmgx4Wl c0W5Z1p5cmDF9l8L/hc959AeyNf7I9dXnjekGM9gVv7UDUYzCifR3U8T0fnfdMmS8NeI9NC+ wuREpRO4lKOkTnj9TtQJRiptlhcHQiAlG1cFqs7EQo57Tqq6cxD1FycZJLuC32bGbgalABEB AAHNHkd1aWRvIEZhbHNpIDxtYWRAbWFkcGlsb3QubmV0PsLAeAQTAQIAIgUCT4b6XQIbAwYL CQgHAwIGFQgCCQoLBBYCAwECHgECF4AACgkQGuaGDlbL0pOWigf/YVTVf3+ZRnzeGP7CjGV1 Wrrxzjc8h8W64NZasV0XLHGFjl5MYwtm9jJ9gbL8Ubtqstey7lYpjOk2fG6YDhY5eptWCpR6 1QqYrioukhCfKbodSk6PnIZcx719nJVK2P7ihdFEN78TavpBwqIf9hGEcKkMpbRFQv1mYvXD hKVwQGY+8bkH/a/pAWmIyD4qMfKCMurH5DexxEt5SYWu5BB5hd/DWyZ0wuZ+F79KMPzLBPJW 5cpdLNbrvenSqFZGJEGhtTp7GFJJr6lTy8VLBArxmFHiY5jGyR45eZEGDcz86FfGgvPnnpi7 aNCc/ROdF7fnZYPh8uZGGjQbd4EYK4xMzc7BTQRTEHtBARAAoWGsNx6g90r8gcNKaiPpJBiK y8ztV2FyV5LsT0OgQBW3vIxt/odtsxVNNjpyS/BNZCyzLAsFc1WrGBzhYsmPN9SGB5/5YTvk zf5YViU5VAsZlj/MRWCZrWtpic4c0A7N4csOYReNtk/q8YB4PIFsZ9A+kTuoZhnu5t5PdfBA 74+SVwKu84+PZk9wDEY1LbFVT8vM42oKsmoswlIhwJ2xuJI/gbk+cMUe0yiRpNjo4Svw4RB8 4B6uFwdRr/PtS7xi2Zqoof5AaQT9YSBpGpKJOe/Qk5MP4PF6Fqq+go89n77Y2kJkwcHaLoD/ GJ+ZDASIiMRe1y54FHOQ1RCTGGpnJLXdKuGhwv3J21pU8HNlq0ASNQMMQmYAwtUWzjmp/KEy I1qkcmjafcxb8TmiaoK8SQN1Zf96fc/sIrZN6Z5oOCEyyCQ0prH/PTA2jlRkKQ487PTGk2JS KU5VuS57Nlk2DrnvjWp57aV9eFAhpnrrJPuGmFz83/Pc8gC0t7N7i7VVHYRcC5naxYB2UoI1 OUkyxpT/HvQFXXVZ3/KmdXMzrx191AggCPWIwUAP+VcaURSYpeDk6/ZVAOVOe1ChqcJisCD7 wK20/OOvJ2AtkWreGu1CZ9zSx7nK/VYdLr34GxQ4bT1G+9rBQNnFSNbX2TJ431Mdo1GCjDeR K4CtSnrNKYkAEQEAAcLAXwQYAQgACQUCUxB7QQIbDAAKCRAa5oYOVsvSkw3nCADhsKRf+rAR ULTpOh5HoLam62ZJZAyCkNqqu/rke5uj5AaaDY/h7BNhBDiDqhhZLTeofGpVVaErPsWN+tX5 0fypsIt9KAhy90GFrtrIZlWuyK4wsoZvDfp9yaRk+lIM58dw/Rcfxn670JaPTFSRPECVn/uL qBhJSkbYlY212YT9fxVUTJe6wIvDLQrQEjrQD/h1FMhfcLhAqsndltRd6DPvTKeMd/6VAxn0 hkoBKhEy5LkWjM9CHppu+bBkQ91/kj2uJQSXO8euonwHHS3c+6N2i2H7I0emcHGu07wuRB2t Dnw/RLBxohffdPZT2kbxuG7lhVHzwVDw5DRwSw8GkOdy In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 4TNPpX0vmCz4LYh X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:24940, ipnet:2a01:4f8::/32, country:DE] List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org On 28/01/24 22:23, Warner Losh wrote: > > > On Sun, Jan 28, 2024, 12:38 PM Guido Falsi > wrote: > > On 28/01/24 15:15, Guido Falsi wrote: > > Hi all, again, > > > > I have some more findings about this, I'm top posting because the > old > > message is not really that much relevant anymore. > > > > I'm now running a machine with head (commit > > b32d49cfbaa0437d08e65e7cd7c82c5951b1a852 Jan 25th), poudriere > installed > > in it, machine is amd64, with an arm64 jail, 14.0-RELEASE, installed > > from official distribution binaries (https download method), with > cross > > tools. > > > > To make sure everything is aligned I rebuild everything: updated > head, > > rebuild cross tools in the jail, recompiled all ports for the host > > architecture and force reinstalled them, especially > qemu-user-static, > > cleaned up all packages for the arm64 jail. > > > > If I missed something important please point it out. > > > > I have made some more tests and I'm getting python failures in > poudriere > > like the one described below from time to time (don't have hard > stats > > but feels like 50% chance). If I get past that it usually is able to > > build all the not many packages, but locks up at: > > > > Creating repository in /tmp/packages:   0% > > > > BTW, forgot to mention last time this worked without issue was around > 20th December. > > > I think this is a bsd-user issue. There is a race somewhere in that code > that causes the hangs. I'd love a reproducible test case that is > somewhat smaller than python... there are bigger races with the newer > stuff and I've not had the time to chase it there either. 😞 First of all thanks for your feedback. It encourages me having someone else with better knowledge about this confirm that a race condition is actually a possible cause! Strange this has not been happening up to mid December. My main and fully reproducible use case is actually mostly with pkg. at the end of the run poudriere runs `pkg repo` to create the meta files and sign the repo. It forks itself (ncpus + 2 I guess, even forcing it to 1 worker I see three processes), and then locks up, with all the processes stopping using CPU (ps output is in my message) I guess this can be reproduced with any poudriere repo with at least more than ncpus packages in it. can also be reproduced using `poudriere pkgclean -u ` If that does not work I'm not sure how to reproduce it in other ways, but I can try writing some code mocking what pkg seems to be doing, not an expert at such things, though. -- Guido Falsi