From nobody Thu Mar 24 03:38:01 2022 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 025D51A28616; Thu, 24 Mar 2022 03:38:02 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4KP9sj6RNVz4pnG; Thu, 24 Mar 2022 03:38:01 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1648093081; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=LZDVRKIz9jUOJxOAdI1E983hJ0pXE3tYupr5Z3322cc=; b=CpT+jgNntyPgo+7uPiZdFos1RqAFrTj2O2wc56yQxHr119edbZWWHOChV9RvAKfwMscC29 4JnlAmexN89mYt4U0hiXZz1HyI4WyPB4ytP8i54Ig00gZKOMaGbM5++A40zzDr16tgGHGO G8Wa9lRMhxiA+Mww4s5ZOHMj+nidUIbAxUucrVBwu2L1dqLxOO24Oll7p6P6dIORgcpyze VA3rfy14Bhwbdc4oVgeCfI5FskANHULlPWzGMjBSLvH0uXYzTfGnuwl/X0SykiDn5Y8ENv 1yrOYulr8gNx5AwuWQ+PhlK1b90pzdwJIfgyVSH2SkQgiZ+tG0yMxynUz1Fe7Q== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id AF457107B1; Thu, 24 Mar 2022 03:38:01 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 22O3c1cx052949; Thu, 24 Mar 2022 03:38:01 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 22O3c1XI052948; Thu, 24 Mar 2022 03:38:01 GMT (envelope-from git) Date: Thu, 24 Mar 2022 03:38:01 GMT Message-Id: <202203240338.22O3c1XI052948@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Li-Wen Hsu Subject: git: e62aee66b0f9 - main - DTrace test: Add a new keyword SKIPCI to gentest.sh List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-all@freebsd.org X-BeenThere: dev-commits-src-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: lwhsu X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: e62aee66b0f9020d712d875a5908b34d07ea0118 Auto-Submitted: auto-generated ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1648093081; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=LZDVRKIz9jUOJxOAdI1E983hJ0pXE3tYupr5Z3322cc=; b=rMCkBM4OpXrkSRI+eHP63G1XCaTPGYde02nTJFSxNmBPz37wLdV5GKDG1NNDvpwsF/hR+H 0jgDmNRrgN8pfkxH3Cm12Q9Bda1Tshj6iIyMLGvyhR5QaTG/zbACrgD8tvZXSP2YQd46wF dtxe1K23v75vSmHtwFQqPzgu/cc2zDF/XB/duMfVMOlTYr7D9oQSC315miUN3/DXs9qlSg Ly1R51HU6Hzp5juPEkDvriUB2dpOmZB7lNjUYoEwuIgEmzXS3sV6acMDPeOM779eBo2Y5o 9m3z8JG99XL6ZklIb6sbnGCFhiO9OIDvgCGRrdbML2rsUO8iZQkM6CddMNYImA== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1648093081; a=rsa-sha256; cv=none; b=Ldh2BAZePhZbG11TIxVP5xcqjPNYyYgqWrWdOrZaIuU59BhITXhUkfKDGxZV77Itymmgr9 uQL4pRjEXPqAjpWxxtxyLK4yGeCuSzY/djphU+LM8tU35cmNMC1D5IuQIJameqNryK3IUn hw4iNWBmvnJ7HFJWJ4s+LPlwFPTuLXpeogy0nwRP3dGGOTy4WhJC0iM+isq1t2M7EduNp0 WPUHNYkVDiN0qvZtUIdKPsNKrnneEZBQ1hsYGUsHeknDOcxLGqbLY/C02n01eyLodEJiJ4 pwFJwe2qCuy5Y9uR1WW7PYGbeqbsGSwesvpvVPW2GL1Nn86+1BBPZDEPnTLY1g== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N The branch main has been updated by lwhsu: URL: https://cgit.FreeBSD.org/src/commit/?id=e62aee66b0f9020d712d875a5908b34d07ea0118 commit e62aee66b0f9020d712d875a5908b34d07ea0118 Author: Li-Wen Hsu AuthorDate: 2022-03-24 03:30:27 +0000 Commit: Li-Wen Hsu CommitDate: 2022-03-24 03:30:27 +0000 DTrace test: Add a new keyword SKIPCI to gentest.sh This is for marking a test case is flakey and should not be executed in the CI environment. Reviewed by: markj Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D34635 --- cddl/usr.sbin/dtrace/tests/tools/exclude.sh | 8 +++++--- cddl/usr.sbin/dtrace/tests/tools/gentest.sh | 4 +++- 2 files changed, 8 insertions(+), 4 deletions(-) diff --git a/cddl/usr.sbin/dtrace/tests/tools/exclude.sh b/cddl/usr.sbin/dtrace/tests/tools/exclude.sh index 50f825769af1..b0b67a86b86e 100755 --- a/cddl/usr.sbin/dtrace/tests/tools/exclude.sh +++ b/cddl/usr.sbin/dtrace/tests/tools/exclude.sh @@ -17,9 +17,11 @@ # 1) and 2). # # The SKIP variable contains tests that should not be executed at all. The -# EXFAIL variable contains tests that are expected to fail when run. Please -# avoid adding tests to SKIP unless it really is necessary; with EXFAIL, tests -# that begin passing as the result of a change are visible in the test summary. +# EXFAIL variable contains tests that are expected to fail when run. The +# SKIPCI variable contains tests that are flakey and should not be executed in +# the CI environment. Please avoid adding tests to SKIP unless it really is +# necessary; with EXFAIL, tests that begin passing as the result of a change +# are visible in the test summary. exclude() { diff --git a/cddl/usr.sbin/dtrace/tests/tools/gentest.sh b/cddl/usr.sbin/dtrace/tests/tools/gentest.sh index 9c34b79f547d..6bad066e5419 100755 --- a/cddl/usr.sbin/dtrace/tests/tools/gentest.sh +++ b/cddl/usr.sbin/dtrace/tests/tools/gentest.sh @@ -83,18 +83,20 @@ ARCH=$(dirname ${CATEGORY}) . $EXCLUDES EXFAILS=$(echo -e "$EXFAIL" | grep "^${CATEGORY}/" | xargs basename -a) SKIPS=$(echo -e "$SKIP" | grep "^${CATEGORY}/" | xargs basename -a) +SKIPCIS=$(echo -e "$SKIPCI" | grep "^${CATEGORY}/" | xargs basename -a) FILELIST=$(mktemp) trap 'rm -f $FILELIST' EXIT echo "$@" | tr ' ' '\n' | xargs basename -a | sort > ${FILELIST} -TFILES=$(printf '%s\n%s' "$EXFAILS" "$SKIPS" | sort | comm -13 /dev/stdin $FILELIST) +TFILES=$(printf '%s\n%s' "$EXFAILS" "$SKIPS" "$SKIPCIS" | sort | comm -13 /dev/stdin $FILELIST) # # Generate test cases. # gentestcases SKIPS "atf_skip \"test may hang or cause system instability\"" gentestcases EXFAILS "atf_expect_fail \"test is known to fail\"" +gentestcases SKIPCIS "if [ \"\$(atf_config_get ci false)\" = \"true\" ]; then atf_skip \"see cddl/usr.sbin/dtrace/tests/tools/exclude.sh\"; fi" gentestcases TFILES #