From nobody Thu Feb 24 00:14:39 2022 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 47F9219E4538 for ; Thu, 24 Feb 2022 00:14:50 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: from mail-il1-x136.google.com (mail-il1-x136.google.com [IPv6:2607:f8b0:4864:20::136]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4K3th93LYNz3ClS for ; Thu, 24 Feb 2022 00:14:49 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: by mail-il1-x136.google.com with SMTP id c14so498225ilm.4 for ; Wed, 23 Feb 2022 16:14:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=sender:date:from:to:cc:subject:message-id:mime-version :content-disposition; bh=8/r9vfFjkC43YjbKOUQkM+8r7LO54E3WPhTlBV8Ob+Q=; b=IkqRzcUoagWKPhCFuxlDhL4458g4l7aJNr+gsxqOO3Sb9S8iM1B+zNWFR9XKaoAO7W 6ihDUGdbIYZKusjJF2DRt24MW65qke0DuTuPsoblUrLWfeDO77ULCvsjFk60KsQv7czB wSKUtDQpDqPoW3dttOCjcNZMfSpxU6ArhXXpptpj05j98LNAU9V9qW3SGVcg7Eb7tY3Q IRDSip1UXDUyKoc0CL9haidPd6Zv3++IZXpOcPYwNVXdzy4Ac0w/OnMMhv7biqzglQ58 JeZOPcQxCgxAmfSbvX5uLgaFAezQOhMF+/qMIm8+FNWnF0/8RoOyKxvvXa39baEeaNF+ dJtg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:sender:date:from:to:cc:subject:message-id :mime-version:content-disposition; bh=8/r9vfFjkC43YjbKOUQkM+8r7LO54E3WPhTlBV8Ob+Q=; b=62WTjHnwZlX+hvi/X+nl7pEgl01XZ3DajvAUucnKplTjbXRppbYilY2QWCjKhe/jW6 JKrk9uzCuX9dSWo7sSTjCo25JBHBWRRdVnM3MKjKITaxQS4/VrHHtZ6GwkSz2xxhFhUz IkR8N3n49+B3GhcbYMvS3L+oEq5fYQDRGIWFsPYKxvWxxfbY6e9916yQE3q7wZ5pRqLc U/ICKHXj4nti1YskVi35STzTKhAdfIyZ55PKqLwKN24HbKbdrPO+4r6/04PFsGwz19Wv e/m6RciLDtRJguz99DrEkm//paFU+tIiYwHSqka0UE4wP2pGkEWRijSsxsxG8fwQ0J0z /XcQ== X-Gm-Message-State: AOAM530NKqtR5nhriExcLOr9gAjw4t2YN7f/iRS4ksgAj5kKnxFnyjH0 YcniwxqzX29OOqZP2xm5Xrsp1luZaCU= X-Google-Smtp-Source: ABdhPJxosPNYY77OcaC21Q53Hot9QC2bE2+nOyVdnwqSx4dKxFl0ITpXb+wFP51NLDNfA6bwRg6HeQ== X-Received: by 2002:a92:c261:0:b0:2c2:8b62:f6f4 with SMTP id h1-20020a92c261000000b002c28b62f6f4mr120005ild.108.1645661682887; Wed, 23 Feb 2022 16:14:42 -0800 (PST) Received: from nuc (198-84-189-58.cpe.teksavvy.com. [198.84.189.58]) by smtp.gmail.com with ESMTPSA id j14sm749311ilc.62.2022.02.23.16.14.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 23 Feb 2022 16:14:41 -0800 (PST) Date: Wed, 23 Feb 2022 19:14:39 -0500 From: Mark Johnston To: freebsd-hackers@freebsd.org Cc: domagoj.stolfa@gmail.com Subject: patches for CTFv3 Message-ID: List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline X-Rspamd-Queue-Id: 4K3th93LYNz3ClS X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20210112 header.b=IkqRzcUo; dmarc=none; spf=pass (mx1.freebsd.org: domain of markjdb@gmail.com designates 2607:f8b0:4864:20::136 as permitted sender) smtp.mailfrom=markjdb@gmail.com X-Spamd-Result: default: False [-2.70 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; TO_DN_NONE(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FORGED_SENDER(0.30)[markj@freebsd.org,markjdb@gmail.com]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[markj@freebsd.org,markjdb@gmail.com]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20210112]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; TAGGED_RCPT(0.00)[]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; DMARC_NA(0.00)[freebsd.org]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::136:from]; MLMMJ_DEST(0.00)[freebsd-hackers]; MID_RHS_NOT_FQDN(0.50)[]; FREEMAIL_CC(0.00)[gmail.com]; RCVD_TLS_ALL(0.00)[] X-ThisMailContainsUnwantedMimeParts: N Hi, I've finished a first draft of some patches which extend the CTF (compact C type format) definitions, making them a bit less compact but increasing the limit on the number of permitted C types, as an amd64 GENERIC kernel has been close to the limit for a while now, causing frustrating dtrace failures. The new format is CTFv3, compatible at least in principle with binutils libctf. The patches add support for v3 to various CTF producers and consumers. The one which adds v3 definitions is https://reviews.freebsd.org/D34360 and the rest can be viewed under the "stack" tab. In particular, the limit on the number of representable types grows from 2^{15} to 2^{31} - 1, hopefully enough to last for a while. The kernel's CTF section grows somewhat. The (zlib-compressed) on-disk size for an amd64 kernel increases from 1.02MB to 1.08MB, and the uncompressed size increases from 2.50MB to 3.31MB. In other words, the impact is hopefully unnoticeable if one's not using dtrace, and otherwise I believe the increase in memory usage isn't prohibitive. With the patches, libctf and ctfdump handle both v2 and v3. ctfconvert is changed to always emit v3, and ctfmerge can merge v2 and v3 containers, always creating a v3 container. The kernel works with v2 or v3, i.e., there should be no problem using dtrace with an updated kernel and an old version of the CTF toolchain. If it really becomes useful to be able to request v2 output from ctfconvert and ctfmerge, it could be added, but I don't have a reason to implement it yet. Any feedback on the changes would be appreciated. Thanks, -Mark