From nobody Tue Sep 26 07:05:18 2023 X-Original-To: ports@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RvrNd6NZ9z4tk6P for ; Tue, 26 Sep 2023 07:05:21 +0000 (UTC) (envelope-from madpilot@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RvrNd5xSgz4LwV; Tue, 26 Sep 2023 07:05:21 +0000 (UTC) (envelope-from madpilot@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1695711921; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=2cFwqoSzx/u2mBkov2sbxUu1HdI0bmJQ85EyQ5voh3o=; b=cq/bUHoQviWD7h2JMAmjBPCZn2Vsp6J5O/HemYPgkbHeeq2gcGXMI9tbZgzBrVIdQxlTJ3 eprG8rjzJXyFDScceZwkNj9M+88FLOJOiH/h2j45i41dTSJIcM+1ssFccodZhiHHxHc21Q o7/YdWp6zcSX6GY7JPC8OJ59z9XEceo3OuAQQAH+PiHWsYBG0Ijl/YIsf4I7zMN4Ncort0 CDNU7SXfSLvGvvfKTRdtJl7gbLvhy1Ook6ldoFQDw4o4R1zzRQi/RpjovoLzPazCj4u2pY mXJG+Z1DDUfy/cxM9TpnQsvlXIuTG58D3NxJahd5rzhnVns4OW4A7XVPqv1big== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1695711921; a=rsa-sha256; cv=none; b=OtQMAcsJHunv2+e5v1yu3p1tuvXTRCnY8qPvgieO4Jpm5dzxuCTZb9tTEiRFxK1vjC8HoE g7kBV8Q/GFd8Z40N4k4PR3cOUSRKqYOB+MtYHgXE3oPhKGgKBXF1/nkFsyPsZTwHPupeH5 1ahXt9mkge+ZuEC9P4uLTJDX1B2ILWNIiFNT9drB2pSH/HzYlf6w+HEEQktQ3WKwlNQrDF AZSJBgK3yuCDlu/0T9jEm6rLMiLSs89ssmPc/bw2ES2m12pT4eSeKYlNJwrBDDUT9uTT7v NlfThNgvZsBXWo667a+O897IiZW6hEyI0jeXqYuRco5GNh4PRFPF2mAlUq060w== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1695711921; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=2cFwqoSzx/u2mBkov2sbxUu1HdI0bmJQ85EyQ5voh3o=; b=AdCwIKiJgM6yvlvdsdvU3biveCAu113o84Tvr5sjNQrI4GygXw/P2ltAr2bttYxBjy+IIb gVoMuXZi57UKybDQNSx1+1pKAc35WuMPybYPM9PS72xRiNuIPz8cRK9dXQtoAqqaHZejy3 1vQR5DYW8JiymKT0Dlns2sJMSFp4zcW+OKjdYHAANmk3bb1OTW/jnE0f3yTR4k8Vvu1Llv W/Js4/0Oe3cYy9Hh53p/Qqe4ehCL2pjTym6rfv6VNo/iXBztjfZiq3PLokdsUOFVqxdZ/f JlAxpELNv2XJYGTaBCyreGkMJMahzeBHtN0Q8gzlG4T6jTwt0vi0j24Ystd9LA== Received: from [IPV6:2a01:e11:2002:4280::13:1] (unknown [IPv6:2a01:e11:2002:4280::13:1]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: madpilot/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 4RvrNd13pVz5y2; Tue, 26 Sep 2023 07:05:20 +0000 (UTC) (envelope-from madpilot@FreeBSD.org) Message-ID: <62a758f5-168d-0205-b4c9-9f980ff00102@FreeBSD.org> Date: Tue, 26 Sep 2023 09:05:18 +0200 List-Id: Porting software to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-ports List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-ports@freebsd.org X-BeenThere: freebsd-ports@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:102.0) Gecko/20100101 Thunderbird/102.15.1 To: Dan Mahoney , Robert Clausecker Cc: "Edward Sanford Sutton, III" , ports@freebsd.org References: <1e05be67-cb15-964e-c78b-e74e714257a9@FreeBSD.org> <11aede54-89d0-9b46-28f8-1931571b8917@m5p.com> <6fd5dbf4-3fc5-a728-d751-d58deeabe933@m5p.com> Content-Language: en-US From: Guido Falsi Subject: Re: dns/bind916 builds rust unexpectedly In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit On 25/09/23 23:58, Dan Mahoney wrote: > > >> On Sep 25, 2023, at 14:41, Robert Clausecker wrote: >> >> Am Mon, Sep 25, 2023 at 02:15:19PM -0700 schrieb Edward Sanford Sutton, III: >>> Poudriere still causes crashes in my experience depending on hardware, >>> poudriere configuration, and what you tell it to build; I have to change >>> defaults to use it and then it is fine. With only 32GB of RAM, if I have it >>> operate the whole build process in RAM >>> (USE_TMPFS=all>/usr/local/etc/poudriere) then I have to limit myself to 2 >>> build jobs (PARALLEL_JOBS=2, though I specify it on the poudriere command >>> line) or I 'may' have issues. Within that I can have multiple jobs per build >>> (ALLOW_MAKE_JOBS=yes>/usr/local/etc/poudriere, >>> MAKE_JOBS_NUMBER=4>/usr/local/etc/poudriere.d/make.conf) which gets through >>> each compile job much faster so no one job is as likely to be running >>> 'alone' holding up other jobs while cores sit idle. >> >> Ports tend to eat more and more storage for their source and dependencies these >> days. I encountered the same issue you had. A simple fix is to switch to >> USE_TMPFS="data localbase" which keeps the source code and objects of the package >> to be built on disk while placing dependencies into RAM. This does not actually >> reduce performance by much as the biggest spike in disk usage is when dependencies >> are installed and the distfile is extracted. This setting helps with the former >> while not doing much to the latter. During the build, compilers infrequently >> write small objects to disk, which gets handled very well by the write cache, so >> having the WRKDIR on disk doesn't hurt. The cache also quickly picks up on >> frequently needed source files, so there's little penalty in having to read them >> from disk every once in a while. > > I really, really wish the ports tree supported the option of: > > "I am building package X, but it requires dependency A, B, and C, which I will build as completely standard, so pretty friggin please, just pkg install them." > > So often have I done the dance of "Make...crap, ^c^c^c, pkg install autoconf...Make, ...dammit... pkg install dialog4ports....make....dammit, crap install bison...." > > pkg, as far as I know, has no concept of tracking the build-dependencies of a thing, but I would love the option of pkg --install-all-build-deps-for foo. > > "Just run poudriere" is a really crappy option when you just have a single box on an AWS t2-tiny install. > There is work in poudriere development to support using binary packages for dependencies. [1] Poudriere already has a -S flag to avoid rebuilding things when a run dependency changes. I see there is ongoing work related with this flag, but it should be usable most of the time, although it exposes to some of the same risks that building manually causes. [2] DISLAIMER: Everyone is free to manage his own machines as he sees fit, there is really no sanctioned way to do that, and building ports on live machines is supported, and I have fixed bugs in the ports I maintain that failed to build properly in such an environment, and will keep doing that to the best of my abilities. But I also must state that doing things like that is a way to look for unnecessary trouble. Poudriere is resource hungry, because building software is resource hungry and will get even hungrier. I started computing on a Spectrum 48K and am still baffled by how we got to a condition where 48K is not even enough for a simple configuration file, but here we are. [1] https://github.com/freebsd/poudriere/commit/a587ae7a3741d56a3991d030a8d08b6e9e619f07 [2] https://github.com/freebsd/poudriere/issues/806 -- Guido Falsi