From nobody Mon Apr 15 00:06:16 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VHnX95x6sz5HLb9 for ; Mon, 15 Apr 2024 00:06:33 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-ej1-x635.google.com (mail-ej1-x635.google.com [IPv6:2a00:1450:4864:20::635]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4VHnX85sbtz465D for ; Mon, 15 Apr 2024 00:06:32 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-ej1-x635.google.com with SMTP id a640c23a62f3a-a51b008b3aeso310058066b.3 for ; Sun, 14 Apr 2024 17:06:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20230601.gappssmtp.com; s=20230601; t=1713139590; x=1713744390; darn=freebsd.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=b0IMDOz2KVBZcWavPie2Cas7+k0Jzb8uUyxrnzDgQcw=; b=Dkt3AKfmWoxP2Y5xZV1GmUJviscvxgodQTmyFgI4JPCoogSETV59MFFHZNOfACF5P5 ZFB0TZccxsrG+kFT+1dbQfL61YAW1V4isaJ5hkKHBiRo9rI4hB8gd6fTMc/HqzaWVR+a 0O2M+h5XIPYjRXf0XUCRPzEdSdAQFgti0j67V3RII9i0tvelMgbkdUIvsdU6YeDmH+p6 eDI7RN8foag18Lld4Dc3nE49HyZ8M79i458KU4mlmr+b009GoUIPXzDLDJGb1ZA43Rx0 PA+fluIVz9FROUbMyn0E7C3CxqQQD6Bmkq2pKzEVjOk6KTyzvzqIW5ulLGHCrrfkAlJz eIzw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713139590; x=1713744390; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=b0IMDOz2KVBZcWavPie2Cas7+k0Jzb8uUyxrnzDgQcw=; b=XskQNFDVV2FgQeeblUeitvde5t+YLdHdtFPUVLNn45+beMwuwNpepymgJxeLHhcTiP 5flZJ1ZzCUjs0f7sHNNRJneClNwEjrLgGcHsp0GD53+dj/JLksa53eNtzS4PDXkI666B ps/6/i7JE7XdL/Vge+/H6QYxCYMx1YHMpDWV7uQN7cEQ7eSLYYA/IpLCGJv4U7ltAI9r lmjwq2/gmVPf5TGd0RLsokD5p2BL2Xgb1TPSu5L4sZWo9pi8L0ImZ1QT9vZQIz6g52MC 8JydqpBN4w8WQC75LzP9TvDhtnKl7bPzSVwWaiXOp3VEWuBib5ejdkxlM2mNiZilCrGd YPZA== X-Gm-Message-State: AOJu0Yzt3JnR78FMc6/VeODze1oT0PMkHFkS+mN3PbOz4W8on65/MQQH gjPDz8E+BwKtraCAvJSJaWUmavDvzUt7GACrGkzIuQgQKnWm3E5FcUXsk0WYdj0rMIE99ubtoRp UEsZgfdGrZ8CY34ywpkpMp77qsjAstRf6NcfBJaOfYc+nWHzs X-Google-Smtp-Source: AGHT+IFDyaVBnNbYIwv5DGQMUm/hN31b0QsMTgSAt2BnH0WWohXulI6qARPSvqihepCgVigyhRWEtbz8c5HMRFjoyo0= X-Received: by 2002:a17:907:770b:b0:a52:5460:a1c6 with SMTP id kw11-20020a170907770b00b00a525460a1c6mr3101349ejc.48.1713139589356; Sun, 14 Apr 2024 17:06:29 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 References: In-Reply-To: From: Warner Losh Date: Sun, 14 Apr 2024 18:06:16 -0600 Message-ID: Subject: Re: Question regarding crunchgen(1) binaries To: Shawn Webb Cc: FreeBSD Hackers Content-Type: multipart/alternative; boundary="000000000000acb33e061617646b" X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US] X-Rspamd-Queue-Id: 4VHnX85sbtz465D --000000000000acb33e061617646b Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Sun, Apr 14, 2024, 5:43=E2=80=AFPM Shawn Webb wrote: > Hey FreeBSD Hackers, > > Note: I originally posted this to the HardenedBSD users mailing list. > I'm posting to freebsd-hackers@ to hopefully learn from a wider > audience. > > I wanted to ping the HardenedBSD community, asking about the > usefulness of crunchgen(1)-built applications in 2024. > For FreeBSD they are quite useful still. The HardenedBSD community can make up its own mind. From the crunchgen(1) manual page: > > > The main reason to crunch programs together is for fitting as many > > programs as possible onto an installation or system recovery floppy. > Floppy is antiquated. /resur hasn't fit on a floppy since FreeBSD 3 or so. Yet, it's still useful to have a tiny ramdisk that's either recovery or simple servers only. The binaries in /rescue are built with crunchgen. It seems that > crunchgen-built applications are not (currently) compatible with a > libc built with LTO due to the recent CSU and libc changes. > PR? Seems is aweful vague. The size of the binaries in /rescue on HardenedBSD 15-CURRENT/amd64 > are 17MB in size. That application size alone makes it impossible to > build a "system recovery floppy". Additionally, floppy drives aren't > all too common on the amd64, arm64, and riscv64 systems HardenedBSD > targets. > Don't take floppy litterally here. Control Flow Integrity (CFI) is a compiler-based exploit mitigation > that we apply to applications in HardenedBSD 15-CURRENT and 14-STABLE. > In order to apply CFI to applications, application code must be built > with Link Time Optimization (LTO). > > Over the past few years, I've slowly been working on applying CFI to > shared objects (aka, Cross-DSO CFI). This requires building library > code with LTO as well. > > It seems that with the recent changes to the CSU and libc, the > crunchgen(1) built tool does not produce workable applications when > libc is built with LTO. With libc having such a huge surface area, it > would be prudent to apply Cross-DSO CFI to it. > PR? This presents two possible solutions: > > 1. Enhance crunchgen(1) to support libc built with LTO. > 2. Kick crunchgen(1) to the curb. > 3. Other ideas from the community are possible. > > Does anyone find crunchgen(1) to be truly useful in 2024? If we kick > crunchgen(1) to the curb, we need to modify the build system for > /rescue binaries. > > My own preference would indeed to rid ourselves of crunchgen(1) so > that we can progress towards applying Cross-DSO CFI and LTO to libc. > /rescue is still the last line of defence against botched libc updates. We use it for rare cases where /usr isn't mounted yet and moving binaries is too hard in rc. Crunchgen'd binaries make the cost trivial. Plus, I have several appliances that are a RAM disk with one crunchgen binary. It makes deployment easy, but I know I'm stongly opposed to kicking crunchgen to the curb. It's still quite useful and the reasons proffered are vague and seem little more than LTO bugs... So I'm for (3) fixing LTO to not suck. The attack mitigation is a good feature, but it's not worth killing resue over... But my comments are explicitly in a FreeBSD context. Warner Thanks, > > -- > Shawn Webb > Cofounder / Security Engineer > HardenedBSD > > Tor-ified Signal: +1 303-901-1600 / shawn_webb_opsec.50 > > https://git.hardenedbsd.org/hardenedbsd/pubkeys/-/raw/master/Shawn_Webb/0= 3A4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc > --000000000000acb33e061617646b Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


On Sun, Apr 14, 2024, 5:43=E2=80=AFPM Shawn Webb <<= a href=3D"mailto:shawn.webb@hardenedbsd.org">shawn.webb@hardenedbsd.org= > wrote:
Hey FreeBSD Hackers,
Note: I originally posted this to the HardenedBSD users mailing list.
I'm posting to freebsd-hackers@ to hopefully learn from a wider
audience.

I wanted to ping the HardenedBSD community, asking about the
usefulness of crunchgen(1)-built applications in 2024.

For FreeBSD they are = quite useful still. The HardenedBSD community can make up its own mind.

=
From the crunchgen(1) manual page:

> The main reason to crunch programs together is for fitting as many
> programs as possible onto an installation or system recovery floppy.

Fl= oppy is antiquated. /resur hasn't fit on a floppy since FreeBSD 3 or so= . Yet, it's still useful to have a tiny ramdisk that's either recov= ery or simple servers only.

The binaries in /rescue are built with crunchgen. It seems that
crunchgen-built applications are not (currently) compatible with a
libc built with LTO due to the recent CSU and libc changes.

PR? Seems is awe= ful vague.

The size of the binaries in /rescue on HardenedBSD 15-CURRENT/amd64
are 17MB in size. That application size alone makes it impossible to
build a "system recovery floppy". Additionally, floppy drives are= n't
all too common on the amd64, arm64, and riscv64 systems HardenedBSD
targets.

Don't take floppy litterally here.
Control Flow Integrity (CFI) is a compiler-based exploit mitigation
that we apply to applications in HardenedBSD 15-CURRENT and 14-STABLE.
In order to apply CFI to applications, application code must be built
with Link Time Optimization (LTO).

Over the past few years, I've slowly been working on applying CFI to shared objects (aka, Cross-DSO CFI). This requires building library
code with LTO as well.

It seems that with the recent changes to the CSU and libc, the
crunchgen(1) built tool does not produce workable applications when
libc is built with LTO. With libc having such a huge surface area, it
would be prudent to apply Cross-DSO CFI to it.
=

PR?
This presents two possible solutions:

1. Enhance crunchgen(1) to support libc built with LTO.
2. Kick crunchgen(1) to the curb.
3. Other ideas from the community are possible.

Does anyone find crunchgen(1) to be truly useful in 2024? If we kick
crunchgen(1) to the curb, we need to modify the build system for
/rescue binaries.

My own preference would indeed to rid ourselves of crunchgen(1) so
that we can progress towards applying Cross-DSO CFI and LTO to libc.

/rescue= is still the last line of defence against botched libc updates. We use it = for rare cases where /usr isn't mounted yet and moving binaries is too = hard in rc.

Crunchgen= 9;d binaries make the cost trivial. Plus, I have several appliances that ar= e a RAM disk with one crunchgen binary. It makes deployment easy, but I kno= w=C2=A0

I'm stongly = opposed to kicking crunchgen to the curb. It's still quite useful and t= he reasons proffered are vague and seem little more than LTO bugs...
<= div dir=3D"auto">
So I'm for (3) fixing LTO = to not suck. The attack mitigation is a good feature, but it's not wort= h killing resue over...

= But my comments are explicitly in a FreeBSD context.

Warner



Thanks,

--
Shawn Webb
Cofounder / Security Engineer
HardenedBSD

Tor-ified Signal: +1 303-901-1600 / shawn_webb_opsec.50
https://git.hardenedbsd.org/hardenedbsd/pubk= eys/-/raw/master/Shawn_Webb/03A4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.as= c
--000000000000acb33e061617646b--