From nobody Mon Apr 29 20:47:46 2024
X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
	by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VSy5Y1FRlz5JwRf
	for <freebsd-hackers@mlmmj.nyi.freebsd.org>; Mon, 29 Apr 2024 22:03:45 +0000 (UTC)
	(envelope-from brooks@freebsd.org)
Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83])
	(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
	 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256
	 client-signature RSA-PSS (4096 bits) client-digest SHA256)
	(Client CN "smtp.freebsd.org", Issuer "R3" (verified OK))
	by mx1.freebsd.org (Postfix) with ESMTPS id 4VSy5Y0Vt2z4vwh;
	Mon, 29 Apr 2024 22:03:45 +0000 (UTC)
	(envelope-from brooks@freebsd.org)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim;
	t=1714428225;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=3RkWvzbCZpgYt1VNgHCsP+6QqfpExEHc/vys9HgSAsA=;
	b=RRp9fGy0yMEzC4spFkzsXdqsrmiena7KVDVtODAiR5WFSPuqYktbCGfUHKSkXXi0Mkl8AM
	J8bLuR5DN4wDp0M7GYrC+0S6HWU6TF0c/9A2MSBsIVcYL0d6C2mNapTh9AC5SFoFKRfT+D
	Sx7CxWEbvTR228YqQiGecuWlW7ITVOURESPKsiVpilknfS9o74jcZgRgiJU5qUkEG+iACz
	+IXPV4+sGiXbz0OCldPOal2h1bdN4vIpFROQw2qKfJc12gi5K2JRr1gcKGcs74lgzgBZ2N
	ACRyFADXhSngXwKFAIHgPhId3zWKh+BK4Jco/2tpJQhquQw1dUvIlEYANfbToQ==
ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1714428225; a=rsa-sha256; cv=none;
	b=WnQsikD3o+cL8jGBT2DJqVBvBBKY7WyFa6Jy3LNWdi8UKWxuCPHTMXOP72o42HueMN7qyH
	fg5nV1zAinNCLVRMSY84ka1KKhWSdAJQZDAMuUdPyZSKl22rAa2393lu8SNNMYpfqur3KY
	1DioYVnCyPfyWiP/+U/JBgt3VUUE61SYjEG5wIktQ9813C1ZRj0ODkpD+GFUT9mOErgFii
	oph4NePepmzRc/VG/x5STZAVJHfrb97RJmfYw210AlZMViiCjaFsTD60wSmKHecW9ikRSF
	+E5WbwwXOEwUrIWBKCtOHTtMVmQ/LMuKJCh47Ar5dEIOQUzy2gzpNGkWi5HNxQ==
ARC-Authentication-Results: i=1;
	mx1.freebsd.org;
	none
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org;
	s=dkim; t=1714428225;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=3RkWvzbCZpgYt1VNgHCsP+6QqfpExEHc/vys9HgSAsA=;
	b=P0BjJ0UdxbmjJOKj3Z1RVa+uWcsKbrqbrZjOkGAsakQEmJnMsGXzuhhxJ3oje1y0CiUWVh
	60aq20nBHcSkZWRALjxxiY/MdUNgP/aqnIr8qB3gnyX6w7/ZCUok1DctgB5+JTfJGoeBNu
	KenIe77KrK+onqpUr851TllN9sbilkrit0N8M0aZpJCCy63g2WeaEisWGoTei9uSpQdJT3
	9rYYUc1ObSqpULjbCFrnhWwO6qqZvw6ZtbdaKoZiMUgaR/Zmy0ueaKEgeeJJpveYnbg6u6
	CNRoOVL3GRmy+mYJo6PcvcdtEYOv5KHUJRPeR/KHSc9NhOPtrSs01H3ei70flg==
Received: from spindle.one-eyed-alien.net (spindle.one-eyed-alien.net [199.48.129.229])
	(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
	 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
	(Client did not present a certificate)
	(Authenticated sender: brooks/mail)
	by smtp.freebsd.org (Postfix) with ESMTPSA id 4VSy5X6zdfzhVk;
	Mon, 29 Apr 2024 22:03:44 +0000 (UTC)
	(envelope-from brooks@freebsd.org)
Received: by spindle.one-eyed-alien.net (Postfix, from userid 3001)
	id 1B3FE3C019B; Mon, 29 Apr 2024 20:47:46 +0000 (UTC)
Date: Mon, 29 Apr 2024 20:47:46 +0000
From: Brooks Davis <brooks@freebsd.org>
To: Christian Weisgerber <naddy@mips.inka.de>
Cc: freebsd-hackers@freebsd.org
Subject: Re: How to run tests without installing?
Message-ID: <ZjAHcluVI2SGh8Ul@spindle.one-eyed-alien.net>
References: <Zi93yxSsEnttr3Ay@lorvorc.mips.inka.de>
List-Id: Technical discussions relating to FreeBSD <freebsd-hackers.freebsd.org>
List-Archive: https://lists.freebsd.org/archives/freebsd-hackers
List-Help: <mailto:freebsd-hackers+help@freebsd.org>
List-Post: <mailto:freebsd-hackers@freebsd.org>
List-Subscribe: <mailto:freebsd-hackers+subscribe@freebsd.org>
List-Unsubscribe: <mailto:freebsd-hackers+unsubscribe@freebsd.org>
Sender: owner-freebsd-hackers@FreeBSD.org
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <Zi93yxSsEnttr3Ay@lorvorc.mips.inka.de>

On Mon, Apr 29, 2024 at 12:34:51PM +0200, Christian Weisgerber wrote:
> How can I run the regression tests on my work-in-progress without
> installing it first?
> 
> Say I'm changing something in sh.  Can I run the tests on the
> compiled sh in /usr/obj, without having to install my potentially
> broken work into the system?  Running "make tests" in src/bin/sh
> doesn't seem to actually test anything.

Generally speaking you can't.  This is one of the problematic things
about the current test framework.  The best you can do for something
like sh where you really don't want to install a broken one is probably
installing in a jail and running tests there.

-- Brooks