From nobody Thu Jun 27 02:27:53 2024 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4W8jCt2J8lz5NJxy; Thu, 27 Jun 2024 02:28:10 +0000 (UTC) (envelope-from rlibby@gmail.com) Received: from mail-lj1-x235.google.com (mail-lj1-x235.google.com [IPv6:2a00:1450:4864:20::235]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4W8jCs1Lg0z4v6f; Thu, 27 Jun 2024 02:28:09 +0000 (UTC) (envelope-from rlibby@gmail.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20230601 header.b=KfffygSD; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of rlibby@gmail.com designates 2a00:1450:4864:20::235 as permitted sender) smtp.mailfrom=rlibby@gmail.com Received: by mail-lj1-x235.google.com with SMTP id 38308e7fff4ca-2ec1620a956so88083391fa.1; Wed, 26 Jun 2024 19:28:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1719455285; x=1720060085; darn=freebsd.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=BAY8qUx6PgfCSuLqwz3s6LIv0Tx0lbOjndPAhWKiqf8=; b=KfffygSDGoKM2ui2VuGCpZxp39HSkzPs8hxVAJ6NlO5FOc4dVLQkXWQmWdd6kTAd2a bJ84sxJkKeADdE1Z5G/T71ekIdne/p1j6Zeb+8ob+PdvdZ8JoW6Ir22VEZ+KS3IyK+Cq d9psTcj0xRWyMRd3VNWKw2ZD5hyZP3OVfhr49gUrxAS3f/nIOoia+Iqyj/Tex+Ct+UFl YvGxZM/51SD5XEXR43bLlWHnx14IxTKXRKu6MDjnnmHdg8mBMjuto7zO8iieXeOjyHi2 NA0TeTxkHczQqwpWJzoYyvK8fSuUOWxaoDP52CABDfRkPIXU3PAfLwFfZsS4ab+wIBzc tpPw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1719455285; x=1720060085; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=BAY8qUx6PgfCSuLqwz3s6LIv0Tx0lbOjndPAhWKiqf8=; b=cPcjBy4+8MmeuaWv21xSb7lwa/R/IU7jFxRrih1QIs0syt1I3IubmEu6xqOn04Lf1L Ru8FTiLTJMtYnfVFaxPDWLLBw+XE6+Amoz65sABwzPYvpkkk4gVsdq30VCNqtFTbl4Kp Zvtvoqwh5iVuulupLKZsOipO8jZ4KMrpoSKvjjV7ovKEUze/gXmIxjs5IrkOit12Tixj dW+XUFiUEs5jART/Jiy1VAYirVwVYXSKkO8fs9x+cZuA24g48isG3YXizAjUJK3kuqyB VLr1l0N3+tMAggzr6dcuWMYrbtbOUJnEC2SNwrFUHBph6yJ83/SZUt4S5CzeyNPa/KbX Xhew== X-Forwarded-Encrypted: i=1; AJvYcCV+21ZjT9gBnW3x5KJ1BzuKEQT+cjpSZ2i8GFIhEsTAOzs8bD1sNcIHCFc44BmxvWnxwMVaIZAxE6Mr/bWObzqvMy738klzrAAKfK13+/jY9E2dYQzoIKefhS6yQMzk7GXcotGV8OJWDJBYOxdqpaKV7w== X-Gm-Message-State: AOJu0YzpKHoNY1MqIb7SUOi6SNNm4w2PCqv+7qQ7Wu/WmRB/SncYYp8s VRvc7SQu4VKMQQNOZngyy5INZNs5ynRw0xkJjeNpZkfSPf5kWHOWX+AKmVPqY5stUl4rclivKjH RsVVgTuM/qY7s2MPl7II3pPc7CqdZ6nPI X-Google-Smtp-Source: AGHT+IHA8jCRU+ir7Ciw5L62ZuxwW7P65LnHbzWf4bYBoXOESn0QOMioUjZ/yfZtXoTt0Rtvvw3Gk7kY9AsqaA7Wvj8= X-Received: by 2002:a2e:2c13:0:b0:2ec:5917:682a with SMTP id 38308e7fff4ca-2ec5b3331a3mr61823731fa.13.1719455284439; Wed, 26 Jun 2024 19:28:04 -0700 (PDT) List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-main@freebsd.org Sender: owner-dev-commits-src-main@FreeBSD.org MIME-Version: 1.0 References: <202406192058.45JKwB77036327@gitrepo.freebsd.org> In-Reply-To: From: Ryan Libby Date: Wed, 26 Jun 2024 19:27:53 -0700 Message-ID: Subject: Re: git: ddf0ed09bd8f - main - sdt: Implement SDT probes using hot-patching To: Mark Johnston Cc: src-committers@freebsd.org, dev-commits-src-all@freebsd.org, dev-commits-src-main@freebsd.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Spamd-Bar: --- X-Spamd-Result: default: False [-4.00 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.997]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20230601]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; MIME_GOOD(-0.10)[text/plain]; FREEMAIL_FROM(0.00)[gmail.com]; RCVD_TLS_LAST(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; TO_DN_SOME(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; MIME_TRACE(0.00)[0:+]; ARC_NA(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; FROM_HAS_DN(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::235:from]; MLMMJ_DEST(0.00)[dev-commits-src-all@freebsd.org,dev-commits-src-main@freebsd.org]; MID_RHS_MATCH_FROMTLD(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; FREEFALL_USER(0.00)[rlibby]; RCVD_COUNT_ONE(0.00)[1]; MISSING_XM_UA(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com] X-Rspamd-Queue-Id: 4W8jCs1Lg0z4v6f On Mon, Jun 24, 2024 at 11:51=E2=80=AFAM Mark Johnston = wrote: > > On Mon, Jun 24, 2024 at 09:27:55AM -0700, Ryan Libby wrote: > > On Mon, Jun 24, 2024 at 9:06=E2=80=AFAM Mark Johnston wrote: > > > > > > On Mon, Jun 24, 2024 at 08:36:55AM -0700, Ryan Libby wrote: > > > > On Wed, Jun 19, 2024 at 1:58=E2=80=AFPM Mark Johnston wrote: > > > > > > > > > > The branch main has been updated by markj: > > > > > > > > > > URL: https://cgit.FreeBSD.org/src/commit/?id=3Dddf0ed09bd8f836774= 07db36828aca2c10f419c9 > > > > > > > > > > commit ddf0ed09bd8f83677407db36828aca2c10f419c9 > > > > > Author: Mark Johnston > > > > > AuthorDate: 2024-06-19 20:57:09 +0000 > > > > > Commit: Mark Johnston > > > > > CommitDate: 2024-06-19 20:57:41 +0000 > > > > > > > > > > sdt: Implement SDT probes using hot-patching > > > > > > > > > > The idea here is to avoid a memory access and conditional bra= nch per > > > > > probe site. Instead, the probe is represented by an "unreach= able" > > > > > unconditional function call. asm goto is used to store the a= ddress of > > > > > the probe site (represented by a no-op sled) and the address = of the > > > > > function call into a tracepoint record. Each SDT probe carri= es a list > > > > > of tracepoints. > > > > > > > > Questions out of curiosity and maybe ignorance: > > > > > > > > How does this work with relocations? Something must be adjusting t= hese > > > > addresses? > > > > > > The compiler handles this as part of the implementation of asm goto: > > > the inline assembly can reference jump targets with "%l" and > > > they're specified as operands to the asm goto statement. In the kern= el > > > these references are resolved statically, and kernel modules will > > > contain relocations for the sdt_tracepoint_set section. > > > > > > > > +/* > > > > > + * Work around an apparent clang bug or limitation which prevent= s the use of the > > > > > + * "i" (immediate) constraint with the probe structure. > > > > > + */ > > > > > +#define _SDT_ASM_PROBE_CONSTRAINT "Ws" > > > > > +#define _SDT_ASM_PROBE_OPERAND "p" > > > > > > > > Is it because i386 kmods are built with -fPIC? > > > > > > I suspect that that's related, yeah. The compiler might be assuming > > > that some indirection is needed to compute the target address, but in > > > this case it's an address in the same function and presumably can saf= ely > > > be assumed to be an immediate. > > > > > > > That makes sense for the "%l1", does it also apply to the "%c0"? Or > > does use of "%c" for the probe pointer require non-PIC? As in, don't > > the _probes_ get relocated, and don't we need to patch the pointers to > > the probes? > > When I use '%c0' to refer to the input operand, the intent is to insert > the symbol name, not the address of the probe structure as computed by > the compiler. In an earlier iteration, there was no input operand and I > just had something like > > __asm( > ... > ".quad " __STRING(_SDT_PROBE_NAME(...)) "\n" > ...); > > But this doesn't work when the probe symbol is local but has global > linkage (i.e., it was defined with "static"), since we don't know what > the symbol name is at compile time. Hence the indirection, and I needed > "c" to get clang to do what I want. The assembler encounters SDT probe > symbol names and emits relocations accordingly. Maybe there's a better > way to do what I want? It seems that this doesn't work at all with gcc > when -fPIC is defined. > Thanks for the exposition and background. I think I'm still confused. I haven't tried spinning up an i386 machine yet which is probably the more reasonable next step, but I did page through some disassembly. Comparing disassembly of dtrace_test.ko with llvm-objdump -rD: clang amd64: 0000000000000000 : 0: 00 00 addb %al, (%rax) 0000000000000000: R_X86_64_64 sdt_test___sdttest 2: 00 00 addb %al, (%rax) 4: 00 00 addb %al, (%rax) 6: 00 00 addb %al, (%rax) 8: 00 00 addb %al, (%rax) 0000000000000008: R_X86_64_64 .text+0x2f a: 00 00 addb %al, (%rax) c: 00 00 addb %al, (%rax) e: 00 00 addb %al, (%rax) 10: 00 00 addb %al, (%rax) 0000000000000010: R_X86_64_64 .text+0x3b ... 1e: 00 00 addb %al, (%rax) Okay, that looks like relocations, which is what we want. clang i386: 000028a0 <__start_set_sdt_tracepoint_set>: 28a0: c4 28 lesl (%eax), %ebp 28a2: 00 00 addb %al, (%eax) 28a4: 21 18 andl %ebx, (%eax) 28a6: 00 00 addb %al, (%eax) 28a8: 2d 18 00 00 00 subl $0x18, %eax 28ad: 00 00 addb %al, (%eax) 28af: 00 ... 000028c4 : 28c4: 3c 00 cmpb $0x0, %al 28c6: 00 00 addb %al, (%eax) 28c8: b0 28 movb $0x28, %al ... That looks like plain data? It seems at 0x28a0 we have a pointer to 0x28c4, the address of the probe in the .ko. But how do we know to fix this up? Looking at readelf -r, I don't see any reference to sdt_test___sdttest like I do with the amd64. Again, I might be confusing things, using the tools wrong, etc, this isn't really my wheelhouse. I'll try to get an i386 vm up soon to improve my understanding. Ryan