From nobody Thu Nov 07 15:16:06 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Xklys6QVXz5c03b; Thu, 07 Nov 2024 15:16:21 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-ed1-f46.google.com (mail-ed1-f46.google.com [209.85.208.46]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Xklys4XKxz3wq8; Thu, 7 Nov 2024 15:16:21 +0000 (UTC) (envelope-from asomers@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-ed1-f46.google.com with SMTP id 4fb4d7f45d1cf-5c9362c26d8so3848433a12.1; Thu, 07 Nov 2024 07:16:21 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1730992579; x=1731597379; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=srf+epAWm1ok6G5Wr2NN54Jeqt3/TpjUZADdF77PSTk=; b=p4/Mp4tOBMvdnrmv4/JC0z/TISzZzSa7Du+FxzaAcHL1NRPIQ5je/BokqLb+xz0nhs uNxsRT4V51ceXHnpuh9mtXjXaQdiYK+M3CAz4P4XKn+lzPRYI7IEnYFhM0ue9NPw+FL0 yHBSh2rfjD9yPMBH6+bNF5IusTA/pn5aIm8NErxkPrsi5NPubrv7Rkkiae52tn9iRqnl 1uZs1nbRsfAaRjLFIotxRd1mwvkjgVQTOtsryzSgUF8nqfxO08KWm/FWo/2p7k8BvY3E cWW2LCVD4RIqJznVof9Eazv0FlM02cHOwb0C72uG61TkrDEab8KbkM4D/+21p2F6E3fq 5HgA== X-Forwarded-Encrypted: i=1; AJvYcCUib1YbZZIdIV7zrNflnkZnWkn8IAJWS3+AsqFI7ChEktUN+W0i/ajNGAqxeh69DOqP/wyoLnM=@freebsd.org, AJvYcCVx0T0gLSCdKB141M9bK9rDHQYS3YKwaQOOLuKcxvUW6h+/z5CEo6fHZ6MY31byFU4j90IG4lOvOaHTwgXQMwFl@freebsd.org, AJvYcCW6CybSKJKX5jNxdpHd8OQ2xchX2ovQkEjcA4A8QgLlfMV+8bj7+snWGbgKbpW+NyjzRyqu8ahOW88u/63a/s8=@freebsd.org X-Gm-Message-State: AOJu0YwID8siPIuuu3I3Svhi+9+gGO+jv43ImCNSgqf+MFaydKXDhJuu OHBUwrliKWiyDlsLoam/GKGy0lom2uKjgXDI79l4hRSOG4f8g8ShzM54cAbZ3Vh8bYS9jT5R2dt M2BMnHLEwIXJC5+iCNEYPoO8UisEn48n4 X-Google-Smtp-Source: AGHT+IFcPUST7VXkyOeDryku6YD4QkGpaB7YCWgPSnyu70xYvCth0a+CsoAtaPA8bZQb7yG23yz2dxXIKmdS9bCWqfQ= X-Received: by 2002:a05:6402:2746:b0:5c5:c2a7:d535 with SMTP id 4fb4d7f45d1cf-5cf08c2e598mr428629a12.16.1730992579073; Thu, 07 Nov 2024 07:16:19 -0800 (PST) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 References: <9fc6fc72-4c2c-4b09-bdb5-122a49b45295@FreeBSD.org> <9975A26D-6AB4-433F-B4B2-515F956BF366@FreeBSD.org> In-Reply-To: <9975A26D-6AB4-433F-B4B2-515F956BF366@FreeBSD.org> From: Alan Somers Date: Thu, 7 Nov 2024 08:16:06 -0700 Message-ID: Subject: Re: RFC: Add required_klds metadata to Kyua To: Kristof Provost Cc: =?UTF-8?Q?Olivier_Cochard=2DLabb=C3=A9?= , Igor Ostapenko , freebsd-testing@freebsd.org, freebsd-hackers@freebsd.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US] X-Rspamd-Queue-Id: 4Xklys4XKxz3wq8 X-Spamd-Bar: ---- On Wed, Nov 6, 2024 at 1:27=E2=80=AFPM Kristof Provost wro= te: > > On 6 Nov 2024, at 21:13, Olivier Cochard-Labb=C3=A9 wrote: > > On Wed, Nov 6, 2024 at 7:12=E2=80=AFPM Igor Ostapenko = wrote: > > The problem to solve here is a well-known one: typically we invoke "kyua > test" and get a lot of "Skipped" due to many kernel module requirements. > And > the pain is to know which ones need loading, especially if we target a > sub-tree of /usr/tests. Also, the FreeBSD CI needs periodic sync if a new > test is added with new module requirements. And other related issues. > > Hello Igor, > > Thanks again for working on this topic > And there are even more challenging cases: what about tests that assess t= he > behavior of different modules and need to unload them as well? > For example, sys/netipsec/tunnel/* or sys/opencrypto/, which involve > loading and unloading modules between tests." > > > That=E2=80=99s a good point. I ran into that a few days ago when I looked= at using the new execenv=3Djail option to parallelise the netipsec tests, = and discovered that that wasn=E2=80=99t straightforward. > > I wonder if an forbidden_klds or something is the answer there. At first = to skip a test if the forbidden module is present, later on perhaps to enfo= rce unloading of the module so the test can run. > > That may not be straightforward through, because module state is global, = and if we only support loading required modules that=E2=80=99s easy (gather= the list and load, or load when starting a test that requires a new module= ), but if we support unloading them it=E2=80=99ll affect test scheduling. (= i.e. we can=E2=80=99t load modules while a forbidden_klds test is running, = and can=E2=80=99t unload while a require_klds test is running). > > That said, perhaps we don=E2=80=99t need to worry about that just yet. It= =E2=80=99s a minority of tests that forbid modules, so those can just keep = doing what they=E2=80=99re doing already (i.e. run exclusively, do the chec= ks in the test script itself). It shouldn=E2=80=99t stop us from making the= majority of tests better. > > Best regards, > Kristof I too like the idea of Project A with required_klds or required_kmods. But how would you handle situations where a user customizes their kernel config to build some feature that's usually a module directly into the kernel? I would think that would break any test using required_klds. But project B is going to be a lot harder. "forbidden_klds", as suggested by kp, might be the way to go. In addition to the abovementioned suites, the fusefs tests are incompatible with the mac_bsdextended kld. I suggest focusing on Project A for now. That would be great even by itself. -Alan