From nobody Mon Oct 30 00:40:57 2023 X-Original-To: ports@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SJZFg2f6Jz4yQvN for ; Mon, 30 Oct 2023 00:41:11 +0000 (UTC) (envelope-from tatsuki_makino@hotmail.com) Received: from APC01-SG2-obe.outbound.protection.outlook.com (mail-sgaapc01olkn2054.outbound.protection.outlook.com [40.92.53.54]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "DigiCert Cloud Services CA-1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SJZFf6NVWz3Fvs; Mon, 30 Oct 2023 00:41:10 +0000 (UTC) (envelope-from tatsuki_makino@hotmail.com) Authentication-Results: mx1.freebsd.org; none ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=fdVs5zbYhfhUMFa0KtUa5dmFtkxzgZykKKGrCtWZUu3kp/o+oklZhCgYC7sfyQK5lNR+O6bRug0GU7HPwV+2AWfmvn63XmDf+E/fG2eEeE7J3tbZvsivdI7ymm3UMUUTOOqrUhsO2N4Ph2gP0x3f8U7pbovJcQFTLW1ZCJtRMDLj1eNU6ILmMG+zyC8719AkGcspIaQFc5nTx8jPbImt++iXNBWCZbyuLzScE559owtp/OeEpuaqw2cfmvsFRBjn5O4bYyKj5iWbje+imPBU+Yk1i7w/hYz2JFFf372lh3chmvAheKAyhjExcDagQzNsSKseMA3o2NycIVTQrBs9IA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=3696DOl0mT6s1W5OdrU3Rsh9cX6O8uMQ3UNZ7mfUEH8=; b=MwSxe4vjiKe2x7RqgiLjdIhoak64vuZ4fZbvEbfsryquoTREy7jFnSeKRKTLkz6IMNfjF4xnxcZo3psZlHtarhzqLOk3AzOxcKzy9cLEpbdxioLmrY9tCiMXfOEaSKQ7DSajxDKGsDUCYUmDlhz/MTZiMvaPChXBuYilSUFJs6h06v/rgba4nSqtG3gHwCV1/2Q1oQlJynMgKpNwwSSTIM5s73T8OnC7ev7BwU2xIeYgtMeIhxwdPTy0uQxMFC/A95RXUR2uTRdzuKKHGWfFi/qtPDiZlqYAR4S7MJrGa9IRPo8hHSZdBA6DsUZ0BFRC6mpsq5GjSyElV4q86M1BWw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hotmail.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=3696DOl0mT6s1W5OdrU3Rsh9cX6O8uMQ3UNZ7mfUEH8=; b=Vt+pOUuCoQsd7QAQhW+zfYwxHh4V0Y6lXa19Odla0sOg0FZM3yhqs2cyz7pr3l5q4t6mX8kAJmlz8j9j1blrZKQd51vM+T5yXsmLat6bWrmJVFGFG+4j9Ud0YogWPsoqZ9Jv0sH3su/mV544WtA7aCyeyjcuusw/N8Bu6Hbi/InfCytj8/FPjlXw8sCAGZsd5jqpiIqzW1/bG0LwqXb+qQHl4O4NdmqDKwKvWg/07ditTxRXYuDA3mPBJAGvhIpfXFjCss7HA9ETnSuZQISndfIDBDh2Morj3Dr9EXgjEoz0YnHfETHoMaOCAzsttx47IY1xaReBuVRCjmG/zXWwkg== Received: from SI2PR01MB5036.apcprd01.prod.exchangelabs.com (2603:1096:4:1f8::9) by SG2PR01MB3690.apcprd01.prod.exchangelabs.com (2603:1096:0:6::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6933.26; Mon, 30 Oct 2023 00:41:05 +0000 Received: from SI2PR01MB5036.apcprd01.prod.exchangelabs.com ([fe80::fb1a:653f:16a:b4d2]) by SI2PR01MB5036.apcprd01.prod.exchangelabs.com ([fe80::fb1a:653f:16a:b4d2%5]) with mapi id 15.20.6933.024; Mon, 30 Oct 2023 00:41:05 +0000 Subject: Re: We need to do something about build times To: ports@freebsd.org, Moin Rahman , Tomoaki AOKI References: <4CDDA3A9-083A-4376-BA74-9431A0CBDCBA@freebsd.org> <3BC6F9F2-70EA-4E84-A482-F14F52F24E24@freebsd.org> <20231029205157.4f388da8c36640cae7e37013@dec.sakura.ne.jp> From: Tatsuki Makino Message-ID: Date: Mon, 30 Oct 2023 09:40:57 +0900 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Firefox/52.0 SeaMonkey/2.49.4 In-Reply-To: <20231029205157.4f388da8c36640cae7e37013@dec.sakura.ne.jp> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-TMN: [dRHFzWBjkHu8UfJj3l4n2zEHMN86UIY8] X-ClientProxiedBy: SI2PR01CA0016.apcprd01.prod.exchangelabs.com (2603:1096:4:191::20) To SI2PR01MB5036.apcprd01.prod.exchangelabs.com (2603:1096:4:1f8::9) X-Microsoft-Original-Message-ID: <30f79bc3-133a-35f5-4c31-8042509adf13@hotmail.com> List-Id: Porting software to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-ports List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-ports@freebsd.org X-BeenThere: freebsd-ports@freebsd.org MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SI2PR01MB5036:EE_|SG2PR01MB3690:EE_ X-MS-Office365-Filtering-Correlation-Id: 97caf15f-0d6a-4b65-c347-08dbd8e0e5f5 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: DpJoaq+E2MkjgyfiZes5Y4e4TCh5+5Cp9rmbNbtPQmaMD8PARFpN33UI/nF0H7b1EdyNwL1iWxvBmokCRP48ErskJC2L6RBoBPiSBbg6KbK1waqXF/+gFjPWEdYFoCBD47vNUYIX/5xcg++ynm1lTX7lBF0VZjk6Jbq2MQI6feS4dW8cw14jmJ39GA/kga847QKC5EFVHWzDROlZsaAv1KLCmMeXd7IPXw2qxF5x5rjFXrPEE0Yesq0Ih7BLKmz93nSjA2jAv0DyLg3WB/zxGM6M5HqnlndU9sYWzqt/fyEIu6OwVyXXQDLhr68IYykQbGfrkWMGN34BSuRTQsHBG4doZF9oAEOYkEEhF6FcXiDa7k5G2PCPqjFcmi5/VaMVGErsZxMNneSygFe2RW1XTMs5YQS8H8flRqgD7f319FuCyTUMrfKzdLhDOl+uyrjGjqsvY1MNuEeVtf0e1AQdk4Svply3ARceQqazslA+ljNvR3aBJ0685mI4XXqit9PVXJ5O7mMqDurNeeSg6DektfIQ+rYqtFcj7ndbns7hVtipyblQSZg9c23LLzfigmp3QUMvOv+BLYrGIisrj1ZrrU8FmORYW89xaxwbO7vanBLXEQ4NX2JGv7Ul4aFkLtmb X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?S3VkRUhIdkkveEhRcWFPRzJXOTlyTHdqUFIxZGF3c0E1OUxaUWEveXhkTGY2?= =?utf-8?B?SnpHa21WVDNBc2Jta1RqQ0t3L1RMTVp4RXVsajM4RzZWMmllMVdaTHpWZjdD?= =?utf-8?B?c3FnNXpsSXZzNHpwSDhuc2p3NjQvNGVEVUtBUG55OHhYZHFGR1ZEV1d6ZmtE?= =?utf-8?B?VDdSOUVCRk5sOGQvM1ZxbnFCdElZZEZKOVZON2taWEE1dFFDT0hucFNFWks3?= =?utf-8?B?ZlBVMktUbWZxci84ZDFjR3c2aVRBMXNNRmR5N3hUQVF0azNYeHNQRVE4bVpG?= =?utf-8?B?Q2RYV0lKWHErWEFMRDE0RDZiRFhCOWVqd3VFWWxUQTVKSkhwU3c0YmJZNEd4?= =?utf-8?B?ZXBkZTRuU1ZjN0FSWU04ZG5iWFJrMk1FNWd4MnFjaGVTc1NpZDh4aDVvU2ox?= =?utf-8?B?SXB5N3lkYjkyN25xVTVvMXk3azBCZlE1Zys4cUFWNzNFeVJRRTBQSHN4SXZM?= =?utf-8?B?MFdnOGtMZWRVZnlmaitnUHRYQ3U1TWl6YzBON05TSXdrOWdMMGgwTGpiV05h?= =?utf-8?B?dEoyZmlHNDlkaENERFVUdVBnZllpVTBQQkpvWmZKbVYxR28zOUE0b2RMd0pM?= =?utf-8?B?Q1RiN1J0Q1lLNytRckl0QzgyUUN1bGVDVkdxc0hyYURCbnNwVENnUEhIeFlR?= =?utf-8?B?RU5TNk02SzBCKysyTTBQSjdlbjNkeHdOdGl3Z2NJczJsQXJtTThuOU5jUVND?= =?utf-8?B?Wk04bTJPaFk5ZWFFblNVY2Y5dll5VE5zNXRFVDEyWVA1L0ZiOE1lMm8xOUto?= =?utf-8?B?T1ltUFpKRkR5bFdGSEtpZzVFSGt1UjdFUWtKb3k1eTBJeTFITXR4ZkJnWUcz?= =?utf-8?B?am5uaG9xQ08wejB2ZUNxMHpzVGNWaFdVZXoyVzlNOGsyMDZEN0cvY3lLaFh3?= =?utf-8?B?T3pEYXdheWhDeDFXcmNvaDJEVnVwMURqcHo3clFYWjVURlB5Qy9zUGJFeHRO?= =?utf-8?B?L0IvZW9rYzBxUmpGOXVyZ3lpejV6UzhWTS9jYU5MMmJ1RFJZaGFVWmE0UW8r?= =?utf-8?B?ZW5tTTVaU1ErMTR4a3hzZlMrU0FpbUxSY1VEY09KQzBVZnc4aWdzZjM3aG5v?= =?utf-8?B?UnFxRER5b0FCZkNXNE5aWUdhbStTcEl4dkgvUUtuOEY4SVFLSmRHdEduYVRQ?= =?utf-8?B?U3VNYUh2VW5yZ3R3bXZpMWdwR0ttS1d5cTg0eEpIcFZLL3NSL2JOSUJEbGdO?= =?utf-8?B?eHVDdGZ1L2Rhb3hWK3hRYk91d0xOc3d0R0dkQkZXcUVEbE5JMWhoaWlJVTNB?= =?utf-8?B?eFBkY2tSb2QrV0FaZ25OZWxHRTZ3TzZFVmNHT24zemdUWTNObUlvMUhlSzR5?= =?utf-8?B?Um5xMlFpUjV1MEhWVkI2eFQ3QlduQWJ3QUQ0VlFEUzBXWUxEd0YxUnFiSC9w?= =?utf-8?B?OVduN3dobHRnWDBUTmJwemx2bjJSZGw5MEJGZXhqYTl1Sjc1M1BPQU9hNnBm?= =?utf-8?B?OFF2QWxEb2xYYmJ5QWpralljaXhLcmYzT0VwYk44eVhsRzQyUHpFT1NOODhs?= =?utf-8?B?T05sdXY1UWRBYzZEbjRMTG1sZWhsS0dIQ2kycUhxcXVVTlUxaWRERVJsMmJy?= =?utf-8?B?MEp3SWR5UmZNMkpnNUhZZmJsWGx0UmdKcFhyRUEyYmdWeElLVWoyMzBYNnZH?= =?utf-8?Q?t+qhxh/OY9cqO/WsugvJVl3hzkpRsRU34HnTsjgtfrz0=3D?= X-OriginatorOrg: sct-15-20-4755-11-msonline-outlook-d8e84.templateTenant X-MS-Exchange-CrossTenant-Network-Message-Id: 97caf15f-0d6a-4b65-c347-08dbd8e0e5f5 X-MS-Exchange-CrossTenant-AuthSource: SI2PR01MB5036.apcprd01.prod.exchangelabs.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Oct 2023 00:41:04.9540 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: SG2PR01MB3690 X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:8075, ipnet:40.80.0.0/12, country:US] X-Rspamd-Queue-Id: 4SJZFf6NVWz3Fvs Tomoaki AOKI wrote on 2023/10/29 20:51: > On Sat, 28 Oct 2023 16:10:11 +0200 > Moin Rahman wrote: >> >> Maybe there are still some place of improvements in poudriere's change detection mechanism specially BUILD_DEPENDS. :P >> > > And maybe indirect dependencies, too. ;-( > > At worst, BUILD_DEPENDS should be recorded in pkg database and some > option to retrieve them, like %r and %d. > I strongly suspect the lack of this functionality affects mis-behaviour > of poudriere on deciding build order and combination. > I don't see any problem in detecting dependencies by poudriere. The various other *_DEPENDS that have been added are only conditional BUILD_DEPENDS or RUN_DEPENDS. For example, LIB_DEPENDS: detect using libname and set both BUILD_DEPENDS and RUN_DEPENDS. FETCH_DEPENDS: BUILD_DEPENDS used when distfiles are missing I think this problem is more of a RUN_DEPENDS problem. It is that ports that waste time when rebuilt frequently have been included in the following structure. [updated popular port like glib] <--+-- run-depend -- [ zero or some port ] <-- run-depend -- [ commonly used in build-time dependencies like doxygen ] <--+ | | | +-- build-depend --+ | | +--------------------------------------------------------------------------------------------------------------------------- run-depend --+-- [ port to be rebuilt as it lost popular port ] It is very confusing :) It seems that any dependencies, whether build or run, will be included in this graph, and that any ports where the run dependencies is broken will be rebuilt. In this regard, poudriere already has a method for pseudo-testing. As I have already written previously, it will be as follows poudriere bulk -j ... -C updated/upstream-popular-port affected-by-upstream-update/port-to-queue Here are some examples... But I can't think of a good example right off the bat :) First, there is a recent common poudriere bulk -j ... -n -C graphics/vulkan-headers devel/py-qt5-pyqt Never forget the -n option when this is performed. Otherwise, disaster will befall the already built package :) This reproduces vulkan updates. This will delete a lot of qt5-*. However, as you can see from the terminal logs, only qt5-gui is rebuilt, avoiding the loss of qt5-webkit. Another. poudriere bulk -j ... -n -C multimedia/aom multimedia/gstreamer1-plugins-core aom has also been updated rather frequently. However, while rebuilding ffmpeg is unavoidable, rebuilding that much will avoid rebuilding gstreamer1 and the ports that use it. poudriere already has the option -S to not detect dependencies. But this is too much. It also makes it difficult to find packages whose dependencies have been cut off. What should be done in poudriere to make this problem smaller is not a review of dependency detection methods. The decision to delete the package should be delayed. When a copy (hardlink) is made in the .building directory, unwanted packages are removed by not being processed. It would be nice to have the ability to stop doing that, copy all packages and then re-examine them to see if they need to be rebuilt when they are rebuilt. It doesn't sound too easy... Regards.