From nobody Thu Aug 22 19:36:59 2024 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WqYP82pGhz5TWD1; Thu, 22 Aug 2024 19:37:00 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R11" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4WqYP81Bs1z4Q3F; Thu, 22 Aug 2024 19:37:00 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1724355420; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=Ua4Qs4OA+m7rCGvRIeI8jvUlKdtkGP28iORzUfIsTYY=; b=hBTk+3EKhrWAZnaaQvzQbxUN8Ho9IxYAZA16w+Fsiiehx+xEOnc/JhHS1JaoTuIKmi8DF3 0bYHy9SEEnS2ezHa64wgQf7rzI/r4zWJS5p0kShpqi4yBCSj83fC9EbahXu7WUxcVQLIvp 45rl0Lxw0SCVkb46iJy9d/eX/ygo4WbHpKjR2xtMEIn0OLsm4hK4ajzaQvaptAKUsdja7h gaY345njrag0i1nT9ePCl8vMFjL+3/uMeEsMib279ogrrerpRiMxnwAy/yB8d+RD1ZeCh1 nLtgb3OaE1giKX8+AeMCZZIp/KbpIoRcg7riTgtI9qcf8xXyaBvfSKRATzCkuQ== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1724355420; a=rsa-sha256; cv=none; b=tgzQu2uXMfQfAA5ag+OctIzk4NZygq9hDhJbrHK7W+ZjZuCykjobi96SZSvGSvexsinWIx eQ/RQQgewjrQGbLYV7DNZCE25cUoAulnnA+ehfUtFA94GUkyOiUAUTwW4IMNFqo3RW5enQ Wki/bfd4n2TNE1rDJcgYnaLXNW33T0IVSmWML6JJ352L71msLTJ/PL0OKhCt8YW25B+Ylt dH97GIJCJiFrIAX0ne+BgNhE8Xlpt7XGeq/LrNOBLIUS7d/JvyyfmzK371NHK0Sk+9kBi7 AWHbIUfoAv3GL7gWPKraztcLfivaPkYrABwm+/gRKAZIecy79tjoya6EKoB0/Q== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1724355420; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=Ua4Qs4OA+m7rCGvRIeI8jvUlKdtkGP28iORzUfIsTYY=; b=eBNxMbhoggO4XhGsNJSD9LoqvGAVmm5b/gvdXQybVlTmoM3DUrrI2hynU5MKoxwfifAI3G Lf+EptCVuEeeFHJf8Perq4yg2c3p/I2xl7tg4b3F7ouO+fbOz9yAds+DbepUxBvHQ62nG7 scqoLPzcvg6umAk/bnmKwtbjB+uqzqP7faMeWvfyHY4O3WmWhO8GfmNPkH6mT7XUlKAHAR fSotDyXtOrA2XNkKekAm/irt2f28BjhgTG1KDUYVmZgyKLQC7nM7T4k545JcNbh6keLt8O KubbL0MsacqQV46NvI9kqewzhUv6O03wQw9fOUzkmGhZ8We2YxNfEu9oayUONw== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4WqYP772sVzh1b; Thu, 22 Aug 2024 19:36:59 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.18.1/8.18.1) with ESMTP id 47MJaxKE034868; Thu, 22 Aug 2024 19:36:59 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.18.1/8.18.1/Submit) id 47MJax47034865; Thu, 22 Aug 2024 19:36:59 GMT (envelope-from git) Date: Thu, 22 Aug 2024 19:36:59 GMT Message-Id: <202408221936.47MJax47034865@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Jessica Clarke Subject: git: 729d2b16b74f - main - rtld-elf: Support IFUNCs on riscv List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-main@freebsd.org Sender: owner-dev-commits-src-main@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: jrtc27 X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 729d2b16b74fa5207a12aa1de190bd930432810e Auto-Submitted: auto-generated The branch main has been updated by jrtc27: URL: https://cgit.FreeBSD.org/src/commit/?id=729d2b16b74fa5207a12aa1de190bd930432810e commit 729d2b16b74fa5207a12aa1de190bd930432810e Author: Jessica Clarke AuthorDate: 2024-08-22 19:36:44 +0000 Commit: Jessica Clarke CommitDate: 2024-08-22 19:36:44 +0000 rtld-elf: Support IFUNCs on riscv GNU/Linux has historically had the following two resolver prototypes: 1. Elf_Addr(uint64_t, void *) 2. Elf_Addr(uint64_t, void *, void *) For the former, AT_HWCAP is passed in the first argument, and NULL in the second. For the latter, AT_HWCAP is still passed, and the second argument is a pointer to their home-grown __riscv_hwprobe function. Should they want to use the third argument in future, they'll have to introduce yet another prototype to allow for later expansion, and then all users will have to check whether the second argument is NULL to know if the third argument really exists. This is all rather silly and will surely prove fun in the face of type-checking CFI. Instead, be like arm64 and just define all 8 possible general purpose register arguments up front. To naive source code that forgets non-Linux OSes exist this will be compatible with prototype 1 above, since the second argument will be 0 and it won't look further (though should we start using the second argument for something that wouldn't be true any more and it might think it's __riscv_hwprobe, but that incompatibility is one we can defer committing to, and can choose to never adopt). Until the standard interface for querying extension information[1] is settled and implemented in FreeBSD there's not much you can do in a resolver other than use HWCAP_ISA_B, but this gets the infrastructure in place for when that day comes. [1] https://github.com/riscv-non-isa/riscv-c-api-doc/pull/74 Reviewed by: kib, mhorne MFC after: 1 month Differential Revision: https://reviews.freebsd.org/D46278 --- libexec/rtld-elf/riscv/reloc.c | 140 ++++++++++++++++++++++++++++------ libexec/rtld-elf/riscv/rtld_machdep.h | 5 +- 2 files changed, 122 insertions(+), 23 deletions(-) diff --git a/libexec/rtld-elf/riscv/reloc.c b/libexec/rtld-elf/riscv/reloc.c index 5ea005a813cb..aa2cc97ae769 100644 --- a/libexec/rtld-elf/riscv/reloc.c +++ b/libexec/rtld-elf/riscv/reloc.c @@ -152,10 +152,20 @@ reloc_plt(Obj_Entry *obj, int flags __unused, RtldLockState *lockstate __unused) for (rela = obj->pltrela; rela < relalim; rela++) { Elf_Addr *where; - assert(ELF_R_TYPE(rela->r_info) == R_RISCV_JUMP_SLOT); - where = (Elf_Addr *)(obj->relocbase + rela->r_offset); - *where += (Elf_Addr)obj->relocbase; + + switch (ELF_R_TYPE(rela->r_info)) { + case R_RISCV_JUMP_SLOT: + *where += (Elf_Addr)obj->relocbase; + break; + case R_RISCV_IRELATIVE: + obj->irelative = true; + break; + default: + _rtld_error("Unknown relocation type %u in PLT", + (unsigned int)ELF_R_TYPE(rela->r_info)); + return (-1); + } } return (0); @@ -187,6 +197,11 @@ reloc_jmpslots(Obj_Entry *obj, int flags, RtldLockState *lockstate) return (-1); } + if (ELF_ST_TYPE(def->st_info) == STT_GNU_IFUNC) { + obj->gnu_ifunc = true; + continue; + } + *where = (Elf_Addr)(defobj->relocbase + def->st_value); break; default: @@ -199,30 +214,89 @@ reloc_jmpslots(Obj_Entry *obj, int flags, RtldLockState *lockstate) return (0); } +static void +reloc_iresolve_one(Obj_Entry *obj, const Elf_Rela *rela, + RtldLockState *lockstate) +{ + Elf_Addr *where, target, *ptr; + + ptr = (Elf_Addr *)(obj->relocbase + rela->r_addend); + where = (Elf_Addr *)(obj->relocbase + rela->r_offset); + lock_release(rtld_bind_lock, lockstate); + target = call_ifunc_resolver(ptr); + wlock_acquire(rtld_bind_lock, lockstate); + *where = target; +} + int -reloc_iresolve(Obj_Entry *obj __unused, - struct Struct_RtldLockState *lockstate __unused) +reloc_iresolve(Obj_Entry *obj, struct Struct_RtldLockState *lockstate) { + const Elf_Rela *relalim; + const Elf_Rela *rela; + + if (!obj->irelative) + return (0); - /* XXX not implemented */ + obj->irelative = false; + relalim = (const Elf_Rela *)((const char *)obj->pltrela + + obj->pltrelasize); + for (rela = obj->pltrela; rela < relalim; rela++) { + if (ELF_R_TYPE(rela->r_info) == R_RISCV_IRELATIVE) + reloc_iresolve_one(obj, rela, lockstate); + } return (0); } int -reloc_iresolve_nonplt(Obj_Entry *obj __unused, - struct Struct_RtldLockState *lockstate __unused) +reloc_iresolve_nonplt(Obj_Entry *obj, struct Struct_RtldLockState *lockstate) { + const Elf_Rela *relalim; + const Elf_Rela *rela; - /* XXX not implemented */ + if (!obj->irelative_nonplt) + return (0); + + obj->irelative_nonplt = false; + relalim = (const Elf_Rela *)((const char *)obj->rela + obj->relasize); + for (rela = obj->rela; rela < relalim; rela++) { + if (ELF_R_TYPE(rela->r_info) == R_RISCV_IRELATIVE) + reloc_iresolve_one(obj, rela, lockstate); + } return (0); } int -reloc_gnu_ifunc(Obj_Entry *obj __unused, int flags __unused, - struct Struct_RtldLockState *lockstate __unused) +reloc_gnu_ifunc(Obj_Entry *obj, int flags, + struct Struct_RtldLockState *lockstate) { + const Elf_Rela *relalim; + const Elf_Rela *rela; + Elf_Addr *where, target; + const Elf_Sym *def; + const Obj_Entry *defobj; + + if (!obj->gnu_ifunc) + return (0); - /* XXX not implemented */ + relalim = (const Elf_Rela *)((const char *)obj->pltrela + obj->pltrelasize); + for (rela = obj->pltrela; rela < relalim; rela++) { + if (ELF_R_TYPE(rela->r_info) == R_RISCV_JUMP_SLOT) { + where = (Elf_Addr *)(obj->relocbase + rela->r_offset); + def = find_symdef(ELF_R_SYM(rela->r_info), obj, &defobj, + SYMLOOK_IN_PLT | flags, NULL, lockstate); + if (def == NULL) + return (-1); + if (ELF_ST_TYPE(def->st_info) != STT_GNU_IFUNC) + continue; + + lock_release(rtld_bind_lock, lockstate); + target = (Elf_Addr)rtld_resolve_ifunc(defobj, def); + wlock_acquire(rtld_bind_lock, lockstate); + reloc_jmpslot(where, target, defobj, obj, + (const Elf_Rel *)rela); + } + } + obj->gnu_ifunc = false; return (0); } @@ -232,7 +306,8 @@ reloc_jmpslot(Elf_Addr *where, Elf_Addr target, const Elf_Rel *rel) { - assert(ELF_R_TYPE(rel->r_info) == R_RISCV_JUMP_SLOT); + assert(ELF_R_TYPE(rel->r_info) == R_RISCV_JUMP_SLOT || + ELF_R_TYPE(rel->r_info) == R_RISCV_IRELATIVE); if (*where != target && !ld_bind_not) *where = target; @@ -251,13 +326,9 @@ reloc_non_plt(Obj_Entry *obj, Obj_Entry *obj_rtld, int flags, const Elf_Rela *rela; const Elf_Sym *def; SymCache *cache; - Elf_Addr *where; + Elf_Addr *where, symval; unsigned long symnum; - if ((flags & SYMLOOK_IFUNC) != 0) - /* XXX not implemented */ - return (0); - /* * The dynamic loader may be called from a thread, we have * limited amounts of stack available so we cannot use alloca(). @@ -285,8 +356,27 @@ reloc_non_plt(Obj_Entry *obj, Obj_Entry *obj_rtld, int flags, if (def == NULL) return (-1); - *where = (Elf_Addr)(defobj->relocbase + def->st_value + - rela->r_addend); + /* + * If symbol is IFUNC, only perform relocation + * when caller allowed it by passing + * SYMLOOK_IFUNC flag. Skip the relocations + * otherwise. + */ + if (ELF_ST_TYPE(def->st_info) == STT_GNU_IFUNC) { + if ((flags & SYMLOOK_IFUNC) == 0) { + obj->non_plt_gnu_ifunc = true; + continue; + } + symval = (Elf_Addr)rtld_resolve_ifunc(defobj, + def); + } else { + if ((flags & SYMLOOK_IFUNC) != 0) + continue; + symval = (Elf_Addr)(defobj->relocbase + + def->st_value); + } + + *where = symval + rela->r_addend; break; case R_RISCV_TLS_DTPMOD64: def = find_symdef(symnum, obj, &defobj, flags, cache, @@ -365,6 +455,9 @@ reloc_non_plt(Obj_Entry *obj, Obj_Entry *obj_rtld, int flags, case R_RISCV_RELATIVE: *where = (Elf_Addr)(obj->relocbase + rela->r_addend); break; + case R_RISCV_IRELATIVE: + obj->irelative_nonplt = true; + break; default: rtld_printf("%s: Unhandled relocation %lu\n", obj->path, ELF_R_TYPE(rela->r_info)); @@ -375,10 +468,13 @@ reloc_non_plt(Obj_Entry *obj, Obj_Entry *obj_rtld, int flags, return (0); } +unsigned long elf_hwcap; + void -ifunc_init(Elf_Auxinfo *aux_info[__min_size(AT_COUNT)] __unused) +ifunc_init(Elf_Auxinfo *aux_info[__min_size(AT_COUNT)]) { - + if (aux_info[AT_HWCAP] != NULL) + elf_hwcap = aux_info[AT_HWCAP]->a_un.a_val; } void diff --git a/libexec/rtld-elf/riscv/rtld_machdep.h b/libexec/rtld-elf/riscv/rtld_machdep.h index fb5f5643efc6..c6600b583612 100644 --- a/libexec/rtld-elf/riscv/rtld_machdep.h +++ b/libexec/rtld-elf/riscv/rtld_machdep.h @@ -83,8 +83,11 @@ Elf_Addr reloc_jmpslot(Elf_Addr *where, Elf_Addr target, __asm __volatile("mv gp, %0" :: "r"(old1)); \ }) +extern unsigned long elf_hwcap; #define call_ifunc_resolver(ptr) \ - (((Elf_Addr (*)(void))ptr)()) + (((Elf_Addr (*)(unsigned long, unsigned long, unsigned long, \ + unsigned long, unsigned long, unsigned long, unsigned long, \ + unsigned long))ptr)(elf_hwcap, 0, 0, 0, 0, 0, 0, 0)) /* * TLS