From nobody Fri Feb 24 10:44:26 2023 X-Original-To: ports@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PNRNK532wz3t5Q8 for ; Fri, 24 Feb 2023 10:44:33 +0000 (UTC) (envelope-from mirror176@hotmail.com) Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11olkn2082c.outbound.protection.outlook.com [IPv6:2a01:111:f400:7eab::82c]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "DigiCert Cloud Services CA-1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4PNRNJ3SlDz3pC6 for ; Fri, 24 Feb 2023 10:44:32 +0000 (UTC) (envelope-from mirror176@hotmail.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=hotmail.com header.s=selector1 header.b=BRl7Y5qe; spf=pass (mx1.freebsd.org: domain of mirror176@hotmail.com designates 2a01:111:f400:7eab::82c as permitted sender) smtp.mailfrom=mirror176@hotmail.com; dmarc=pass (policy=none) header.from=hotmail.com; arc=pass ("microsoft.com:s=arcselector9901:i=1") ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=mVSQHPsJBo4C4Z/yn2jcFgnl1R7GFlo0Q1ZPkCWtvKoqQv70Muy8xRnZy780WPaHCjpKwQeqRJuNr4oLTGGzgtIEDtT8cr3Cw5UWKpkVyzuJfIBl8C3ft2c7yBP7iGkjR/4b0dSTzKXh4akJ+BRgemA4JZ6il9TvRiYQvHhk2eQGrlnF9U5PkyY/XyoTQ6Ur9tZ79EQnsVBjleZAeAu8es4y5gsSH4Rlu+eBct+tl06ax+duJDpJ1PJS+ep9PjcnOrL6XWMeHbfuSMRjxNNNQSsvu8hdEelFhj4PN7xhg/X46dK29cAj+mdfP4mwJi6xzOKfT2IdzxkPjk0h+iMTrg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Y8y6JDSLalnKfig5irrYE29RkONwrrZwyh7mv6v9qCU=; b=EK3SDpDFWL1PGVUTd6FbeF4A/qUtqxkf80JRzka+Q+MT5AJTEIRhIP1h6rTtnx9PRk+I++c+8SItsiE//Cf2MF5/pKX2K2MzkEvNsf7iZDZHIyPVMCH2uz9YVLlQi525pe6SnIpCKL0A5j897QECtiSM6/bdjN8Rb6mb99m1Y728v9zMlrqk0rFGiTyWdlk7zMo9xGou0z1QwYrPxISilGh6vZdCgeqC6qih16HrtqXe/+pq09OJewarUiob4DCqXXFWJ1VG60xlvcMLLTplWwDt7WC7uwBQcXCh3aJ+XRlTzymXhcV1Hdur4CIgVUzO84hYsm25go8+hIryI9jY6g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hotmail.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Y8y6JDSLalnKfig5irrYE29RkONwrrZwyh7mv6v9qCU=; b=BRl7Y5qeTeZ25Hql8JVmYUhP6byK7+OY6hoi8plROctXJBvPE/mlVwHSNkvHnNh0M+1X0a5K8VOfeixSvIm+jDXiQwmEvUpJ2VyI2cN/CnXfiyvI14c78zif+r4HNHdnf6fP5Rcg96kXyxYAwt6n3YL88kZnaGHLQg+y6vS5ctjlOwbd+6A3widN1TLOmjXhOwsBJP0SEBZ3eba+hp+ZP89GjTDn0xK1amr9k33B+32Y50Z0FPGXF5N8mPDLHUKs0DO+gUHFTRW8gfshSxm+i7lqFZdjvx1hBCwx5aJOTA4BVTlsxH251xWgO+m57rqNq5pUJ2jtQ/0zTU31Cx3dyg== Received: from CO1PR11MB4770.namprd11.prod.outlook.com (2603:10b6:303:94::19) by DS0PR11MB7482.namprd11.prod.outlook.com (2603:10b6:8:14a::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6134.21; Fri, 24 Feb 2023 10:44:29 +0000 Received: from CO1PR11MB4770.namprd11.prod.outlook.com ([fe80::ad84:7cc7:dc69:fe94]) by CO1PR11MB4770.namprd11.prod.outlook.com ([fe80::ad84:7cc7:dc69:fe94%3]) with mapi id 15.20.6134.024; Fri, 24 Feb 2023 10:44:29 +0000 Message-ID: Date: Fri, 24 Feb 2023 03:44:26 -0700 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:102.0) Gecko/20100101 Thunderbird/102.5.0 Content-Language: en-US To: ports@freebsd.org References: <612cb315-7f3e-599a-ef47-cf2762939da3@fechner.net> From: "Edward Sanford Sutton, III" Subject: Re: Cache go packages to speed up builds in poudriere In-Reply-To: <612cb315-7f3e-599a-ef47-cf2762939da3@fechner.net> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-TMN: [AUiVm+bfyfjUFXKgeR6yDpQajlmbkxZt] X-ClientProxiedBy: PH0PR07CA0056.namprd07.prod.outlook.com (2603:10b6:510:e::31) To CO1PR11MB4770.namprd11.prod.outlook.com (2603:10b6:303:94::19) X-Microsoft-Original-Message-ID: <1c530f2c-6a13-9c46-0e35-63b9adb49e6d@hotmail.com> List-Id: Porting software to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-ports List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-ports@freebsd.org X-BeenThere: freebsd-ports@freebsd.org MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1PR11MB4770:EE_|DS0PR11MB7482:EE_ X-MS-Office365-Filtering-Correlation-Id: 08223e85-aa9f-4305-af20-08db16541afb X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: aHB8nL9IYR5oxP89toXWRopR/FSyHxWaGIoribqDFAFfI6cokjrlqO2dBaJnLHHL/SSntMSDnEu6tH/AkkNUguHb7WHo2wWI1iDU3sh+VMy4ZoWl/kh99H7M+fMOFQJJ4A/Y1QgyOGWgMjIRs5N7rh5ldCUhcAdzB12g18fnCiwUY6jzIyDMZ4FalQelX80DxUyQUygV/ml1M3cTCWGZtdWBhFB0h0EnfhYXPNrUJtyZ/GsK9WrFeTd91CpriVOR1o2+vpsnFfO62eQiXieKKM29r68VS0e8TEkJDa9YbayKqf/mWbftc6TXJc4ZIdDA2sVhrblm/0388jJcLBsfQoVwBqOy2nRbm+nx2oWtbwlvITQgz6dGEQnvGoHai3sxZ3tnxyfuEijhPiLSCuIzO+JpSh8+0LVsUr4aG7lmaZB/JTOcFqPQuH7kfzUvrB2xH67L+HviWi0DYFC24IiffHkVJKyQ77GdRvu6FUNG8+fci2R5qASKDj+Nsy1wQYG9cMC8rYjgaUfkC/R9IQGHAddE2ZiIvttzG7ENm3rj/pooH7pdV84nmt3COX4v0EqWHVZUEm0p1a4OTLVnbfQWDyDQRHyD5s4Oy3ddlPNQ3ThVUqB9jHDd25iWQRfshkJg3u4UHs/Fa7HjELzxXJ8jgA== X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?M2grRmc5V2NnZGlQTGNaeWhEWWlZNmpPc1I4UTFrY0xBUTFDN0ZlM3JmT1lW?= =?utf-8?B?UitodVFLZ3NodVVLZlkwOTNQa2ZnekRMSzgzbW5PNmJmSEFBeEQvQjNRYzgw?= =?utf-8?B?Wm9LM0xVSStQR1FTTmN2cUZWOHhqL2J2bDU1cGFIOGNJVDN0MU12aXlTb1F5?= =?utf-8?B?cTFTb0xud2ZYZmZUeGRvSk9vQ0VIWFY3TnBnUDF6bkJtRTAycENocm9nZ0M3?= =?utf-8?B?M1JBZy9oa1FWdXRvVmc0ektVOElTZ0JGU2ljYjV3TjZ3VHl3WGoxaGxCSnRK?= =?utf-8?B?ZzI5UTQybnNubC9OZDZBQ1JRY0hyd2IzbFhrVFZyamhYSHRIbkorbENweXVR?= =?utf-8?B?WXJtLzEreDQ2czRiVVVjZUhaaHFGdVZhTnNHU3M5UDg4Z29Cck1CNUNVWVNr?= =?utf-8?B?MkpwTzIvbkkvWGNWeG8xZVAvN3JlTjNueEVVMXFoYlA4b0lWa1NDOFRLS3p2?= =?utf-8?B?cFlxdXN3eDliQmJFMUMxMk14YUpORkZxUzhDSWxEelZ3ZTZsWHh0QUVOMTBE?= =?utf-8?B?Z3lTRlZqNHM2T09SbjAzenJTUXpJYlNVa200S1N0Zm8rUDFjU3VqaU1uRU1s?= =?utf-8?B?Z3kvaDZiTk9EM2N1RWhiZ1E3eXVqbmcwUU96WkFITitNNWdNby9WSEpENWNu?= =?utf-8?B?OEU2cjFxandIZ2YrTTNQeWt0ejJNbm5Jc2xXZ3NHd3J3aHVjc0RZcW4rT0Fn?= =?utf-8?B?RUxaY25TcFhqaEtrZElETnBnejlJR3c1OGRQcTBBTnZ0R2EvUnptSFN0ckVB?= =?utf-8?B?bS9yUWVCSkg1TWh6dkVuclZRbElIaTNVMkJRcHgrak9vUVRmRExrNDJ2SFlQ?= =?utf-8?B?c2l5alFud2VTQkIwZHJVUDZWUnBkVXNXMVlDMFdJUkxKbVdzVUMwK09qWGlF?= =?utf-8?B?ckVCQVorS3JlZjk3ZjhnVlRZQkVDL2gzcElOWDVxRGZqRUI4TFZRQm5yYk03?= =?utf-8?B?aGpLSkdWYVhDQ3VLazM0Qi8yNTZRMVVvL3FZOEd6b1Nia2hHVW54eFltK2pE?= =?utf-8?B?Y2REdm54Qk5NODVuNzh4WE1mdmpTMXArbUFXdUJFTjc3dU5hRXhEOWYxTHgv?= =?utf-8?B?TldZay9tc0RiZkw3KzZ3WmxnSkljRjZZY2R2ZEJIeUdCbmowRmpEekpGWnhy?= =?utf-8?B?YmhjMGJqQU5EeHM3bHV3ZFNnTURScTJENGFOSTM3dUZGTEpCTXNTd2pLR2F6?= =?utf-8?B?V0Rwd0VWUkhra3MwZjVPa2NkWkZtc2dqSWQ5OHhDbkI4Ukl4OUV1M0tMWFRw?= =?utf-8?B?Q0pBRnZvZDNVZTZvamUvcU1hOUVNZ1lBRno5VWRnM2ltaWlHSEJQU3FHL0dh?= =?utf-8?B?RFhwbmgxNFZsbFVyNTR3d0I4WUxVeCtnUHBjUnpPR1l2M0s1akEzZXBYZ0x3?= =?utf-8?B?TUErQ0pLa21ISVBwN1lqS3NEVmNWa0pDNjRta25SbkJKWFBVTVBOTEN1a2Mx?= =?utf-8?B?Z3YreStOVW9OWEJMUUd5dWxqZ3hjTHZKUEhwdkNnZy9mY3N6WFNUSWh1Y0hP?= =?utf-8?B?TzNKY08xdWROajV0RnVWdzJaT0NteWdMdlliMWV6TERFT1FnR2JLQ2VzTTUz?= =?utf-8?B?ajQyOEtpVUhaZTZaMkE0K2Z2c3pFUmtMbFJ2SHVLQkhIRm9WaXkxYnB0VWZ0?= =?utf-8?B?SE16WjM3eUN3ejIwZVhoZlE4K3pNZ002TVZuSFI0WUV1STRMMnZEM1YvK29B?= =?utf-8?B?c2hzWW9WUUsrQ3RScVJnRjFHWWs2Nm9YRitjejNHUUFCTDhVQ3MrQ2p3PT0=?= X-OriginatorOrg: sct-15-20-4755-11-msonline-outlook-e8f36.templateTenant X-MS-Exchange-CrossTenant-Network-Message-Id: 08223e85-aa9f-4305-af20-08db16541afb X-MS-Exchange-CrossTenant-AuthSource: CO1PR11MB4770.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Feb 2023 10:44:29.1958 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR11MB7482 X-Spamd-Result: default: False [-1.81 / 15.00]; FORGED_MUA_THUNDERBIRD_MSGID_UNKNOWN(2.50)[]; ARC_ALLOW(-1.00)[microsoft.com:s=arcselector9901:i=1]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[hotmail.com,none]; NEURAL_HAM_SHORT(-0.31)[-0.306]; R_SPF_ALLOW(-0.20)[+ip6:2a01:111:f400::/48]; R_DKIM_ALLOW(-0.20)[hotmail.com:s=selector1]; MIME_GOOD(-0.10)[text/plain]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:8075, ipnet:2a01:111:f000::/36, country:US]; MLMMJ_DEST(0.00)[ports@freebsd.org]; FREEMAIL_ENVFROM(0.00)[hotmail.com]; FROM_EQ_ENVFROM(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; RCPT_COUNT_ONE(0.00)[1]; DWL_DNSWL_NONE(0.00)[hotmail.com:dkim]; FROM_HAS_DN(0.00)[]; DKIM_TRACE(0.00)[hotmail.com:+]; FREEMAIL_FROM(0.00)[hotmail.com]; TO_DN_NONE(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_TLS_LAST(0.00)[] X-Rspamd-Queue-Id: 4PNRNJ3SlDz3pC6 X-Spamd-Bar: - X-ThisMailContainsUnwantedMimeParts: N On 2/23/23 07:01, Matthias Fechner wrote: > Dear all, > > is there a way that go (that is used by ports in poudriere) can cache go > packages? > I ask because if I e.g. make a new port for devel/gitaly which uses go > and I make a test build, it takes about 30-45 minutes to download the > packages. There are upcoming changes to poudriere-devel that are not yet released that should impact determining what to delete and build as I understand it. There are bugs like https://github.com/freebsd/poudriere/issues/954 related to packages being removed when they should not be which will hopefully be properly addressed in various ways. > And it seems that it is downloading the packages again and again with > each testbuild. I don't usually use package downloading as a step in poudriere runs but maybe see if not using packages with poudriere steps makes a better workflow. Poudriere normally only downloads distfiles if the file doesn't match anymore and goes through the full build sequence for dependencies starting at the first dependency that is changed from a ports tree update or changing build options for it and begins recursively reprocessing all ports that depended on it. A rerun of those build steps won't happen unless a dependency is incorrectly entered in the ports tree or I force a rebuild. Some ports are a long and resource intensive process to build but maybe it would help as a replacement to an 'always-download' package issue if the machine is fast enough while internet is slow enough. If not, my next idea would be to do a poudriere testbuild -i and just stay inside that jail doing multiple builds, installs, uninstalls as a test environment. I don't have experience with it but maybe an alternative like synth gives an environment you could compforably use, or just setup a jail or virtual machine where you can manually run package installs and ports tree make commands in a clean but isolated environment. > As I have a really slow internet connection here, this is really very > time-consuming. Maybe there is a way to set up a local caching server to point poudriere at instead of the real package repositories so even if it downloaded again, it'd be downloading from your own machine instead of reaching the internet each time? Not a workflow I normally use so not sure what options may exist. > Thanks a lot for any tips. > > Gruß > Matthias >