From nobody Tue Oct 18 17:38:41 2022 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4MsLgr3xGwz4gHsD for ; Tue, 18 Oct 2022 17:38:48 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic309-21.consmr.mail.gq1.yahoo.com (sonic309-21.consmr.mail.gq1.yahoo.com [98.137.65.147]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4MsLgq2nFZz3cr6 for ; Tue, 18 Oct 2022 17:38:47 +0000 (UTC) (envelope-from marklmi@yahoo.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1666114725; bh=U6xpY4q7LSkJzZ9M1sa/XCKpMCdW3uOec3K2jyNwM8M=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=eu3XO6AROt1UVfUlIJFOv9MUSrX0Gs24IGoRh3BncZ64wp8DBRpYSoDTreAbshTP8jgbzpLfGM4sMoIM/dgzXaCluWUF9XfI8N3t1blEDbmwSQkMgwRIwVuqX14k5hWlWGr2Xqhj0uxc6z8CG0WdP6pbbRemFASniyI8+UkkrPsHupOd/7hLgDwod/I6DgCly5o0BpFtt0LbczNbVQWCfGt82gk0xl35z1xaeNvbnaeVVteo5NFXPjweusR49WpAvPhx981T8CWmz5d+WtPMfBFfaFQE93PFJrsqKUofmgrgWY38HRKJeVFrgo9PwyXSB4ZBi4dP1YIeWvIh3OEdKg== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1666114725; bh=8jmKM/WFZWojJgJP8wlrCoiJSbaofiNRIk0za67EmFr=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=bM4hS4X4VZU0krwTL99tn+j1qo09O5zlN442OnZVjznpRopswG+sJ9wPHi3DMgivkgaVu9pfT+azygmj0rL1yFiy0uPNF5TpYugW9PkyGVXLVEP+0UY7n6zVCufvv10ShvaHJ7D+S1NqHFHdCYhSUhPagEVM9ii3rv7+foGYzsx4d5G5otv0GDBCKVQMKheLoYR7+Kx4MdzhYMuFnyWQZh6EYvc1ijHFwdE8nsErsQfLSTVH9GamHuKWQRTKwxSYe7velDd3KwWt/9fii4GNvqVlz8X0B0tZF+Ccr3ex5bJMsMt/xjyIJ5JS3htcRmuhSsJVcFBZp9aAExD3lfW3hw== X-YMail-OSG: G32cs_8VM1kfwkUwqFnVbe3vIl2Yjji7ymoEMcwCzzkOszpaZgLnAABcGHX3R5p 3Ii9shBEqK5uxaRMkUNjyZsJLjcjxFXKFV8FTzU52FNYVjH07bj4aISIUBfLrMMTq2aOrEFSToIt Lxl8NEaqP7Rh9nTtpahCq8K.xWE94YUwuV5wdOMH7qFD5VOwn_mDw8j1rf6LEpO3wjbydSb1gM4R tP8T_4uYHRNleRC96WB3.v_VNkXQ_1VNODmpBT1UAtnSwhZfmtW33N6J.NrcC8FmVsTU8BjSLXPy dVX50QVB_8Q3ZU7O.biBoYRcBFe7otwx2PqapYDoOATHddG_3TFQ0ON4YVXYvqmFF7057msOlaL3 c8QAMXvOIQl9xCTtCxbKrBrHe1N29oHsOHNKcfOcqFJOO2mvqB0tW8qXlu5KfDheNSFraAO5d8_i n0z8gX8LXtfB09XBWM8LbODERKmPU4cI4scyzgpAAE929MLrxp73yeq.1GvCjM8PBCy4xluUdvHy r4wkJlugSrxQ2hyMPDBA8loDiwP2mMctQGhEQBqvwyp1OrYJj3Rs3pAWnBfE2ICAquWucVKF4v_h ncSq8GFmnj2_l42HvjCuK5sXYW_7lZ2u_XhWw0AyOghiivjpCm.HMljQvrqzo1f8.IUjBdAL1XDm o__Jl0RLrj_EosckxdidY6kTReWQF4fseAntW7JYM.3KOyHn4T_s0_RT1Ih6ExyxrGOrpqkyC05G 5a.l6_P59e0A2GztGHY54jaxsrTun3y412WtXqIbQ7ASNUaS3Mlg3T6OMEh39PrunDI53J0iUeMl zg5XM1P_eKb9e2D7uIOUY_JHOD5Ls2tIZgEy2h3uBdu30BU3EY3vM8ZsVltatY0GRsOSb_Gla9c8 HJjlJd_GL5GH1wMFvHClmXcG2p_W3lbRPSmFR6Bnfqr5TUS_2VMBpsd8Rd.jm3N_vldBEoLCCzJd TgpeRbENS9F2u6NmnwTFri17PGFV5XH.oi_yC0erdBjTZxqVtgY614G9.mn5_wJlQuH6niHuk_vz qAIr1Nm0.2fU1Hineb3TtT.gJPSk1zm7KdqmoQI70uvhFXESy1OG_0AcQFN4.rIlFvx.Y7JNR3pv ECh0t.6L5lg1DQ4sROuIC07X3CBnOvpSfThpqoPlBriRbpIxuzJsh4iTroPq4wk7L3M3e5gqi0Sn D5dLuAeUhWn7hPvHC17dEJERRpSRqUXjy_gS_cknN8tLqHjo7_5G8mVQRO5D6rsbrFGcr1U7O3sD 9Jug4jIP5DuLFZ3PY76rZcygT2ViB.ai1RztrRYa.b1opeHeX7cxnLhxO7waadgak_Cron5CJtMx KVwLVfD8MLlmvZR9E9LaVeU0owdd3NeGcr77jKYDcY8_IS416Svvr3n9IsLT3l2Ts8aDQ2JSd_8q uh1jQehsZWiHJfyfXRmt5TVi1X0o1Nk2i9IMaYgmr26McJYBxxdUkOpqvYKeYAvY2k8IwLTVYooD otf2nCpGeJuAW79lgTnZTno5F2Cdjbgq9uawZEAIqjCWFgZH2A83Egx2bkk5wkDf2FK7n7dZULOy pB2DPrr6D4cUzQY_Y_zSDD5PA3DWMs1j_hmj.4xJtqGXZYa1XqU6bd.aUCbse22rKA6SknsgJ_d0 TeJQxL_fOd_xleDj4q0ckpoGaUxzeY2EReOa5OlDRU4Ikynf_kOAJZgZ8zOtwIEPYlr12Lm7XktJ o0s7qCqDfzW6.mQULEWK4pquAOuJyouFmEmiqpRU2utdIEswlPLhAyyVFC9.tTgAZQRgFE4cqGTG epS1lBG35HP7d.gqdnueMHBeIpwnEA3j1k7yuQS9ObCX7pgJxifHj7F0hUzHhMAR3NtEMleZxDs. lquFL.1ELZ3Y6AsTQcLfKwcg5HZXSf844RjVhzl2sFDMY5W4fMqXQhtfGirVaDffopcMZv_5ArA3 99GrRYMyVTdkCxoTgL85rY2RWOVEA2gfLs6RoMHdegkGz0J5Us1K9oG3ysoJJoWaC.TLWeiB5_i6 LisFOEKcJD5Svyccm8rl_SUp8ZwjCBJZvIhE8A.zUO_5wNEi4NjvazPLcsnCTxdWFCav1_Q4scAj oLpNYtCGIToPya_XFEJZ0n4S2r87JNaRtMB.PzPx3sn.ej0v1CxMY5yMnp5V1HAvedXI2l2Z2hE9 VPnt03W4f1FByYjyAqE5tzJoGLLyiMM6xzo.qp5_Np4qBrQuXCT59zjLiEe5rSL8JMexoRxEe9O_ cSVVfiorgITrIWCn9FD9561oichMPcLWTghow7qhdjN1yePyJBhptnMLDBxXCy_aDwYZkgWszr53 4rQSE1dc- X-Sonic-MF: Received: from sonic.gate.mail.ne1.yahoo.com by sonic309.consmr.mail.gq1.yahoo.com with HTTP; Tue, 18 Oct 2022 17:38:45 +0000 Received: by hermes--production-ne1-5db649d989-kp8mg (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 3d2c83cf779aec0a1e4010f17f44c2fe; Tue, 18 Oct 2022 17:38:43 +0000 (UTC) Content-Type: text/plain; charset=utf-8 List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.120.41.1.1\)) Subject: Re: Running armv7 on aarch64 From: Mark Millard In-Reply-To: Date: Tue, 18 Oct 2022 10:38:41 -0700 Cc: freebsd-arm , Brooks Davis Content-Transfer-Encoding: quoted-printable Message-Id: <3CA3B1F4-46CB-4C75-8451-AF2CA4E9F74B@yahoo.com> References: To: Kristof Provost X-Mailer: Apple Mail (2.3696.120.41.1.1) X-Rspamd-Queue-Id: 4MsLgq2nFZz3cr6 X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=eu3XO6AR; dmarc=pass (policy=reject) header.from=yahoo.com; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.65.147 as permitted sender) smtp.mailfrom=marklmi@yahoo.com X-Spamd-Result: default: False [-3.50 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; MV_CASE(0.50)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MIME_GOOD(-0.10)[text/plain]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RCVD_IN_DNSWL_NONE(0.00)[98.137.65.147:from]; TO_DN_ALL(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; FREEMAIL_FROM(0.00)[yahoo.com]; RCVD_TLS_LAST(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; MIME_TRACE(0.00)[0:+]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; MLMMJ_DEST(0.00)[freebsd-arm@freebsd.org] X-ThisMailContainsUnwantedMimeParts: N On 2022-Oct-18, at 09:53, Kristof Provost wrote: > I=E2=80=99ve recently discovered that I can no longer run an armv7 = binary on my aarch64 FreeBSD machine. >=20 > $ /usr/local/poudriere/jails/pfSense_plus-devel-main_armv7/usr/bin/id > ELF binary type "9" not known. > /bin/sh: = /usr/local/poudriere/jails/pfSense_plus-devel-main_armv7/usr/bin/id: = Exec format error > $ file = /usr/local/poudriere/jails/pfSense_plus-devel-main_armv7/usr/bin/id > /usr/local/poudriere/jails/pfSense_plus-devel-main_armv7/usr/bin/id: = ELF 32-bit LSB executable, ARM, EABI5 version 1 (FreeBSD), dynamically = linked, interpreter /libexec/ld-elf.so.1, FreeBSD-style, for FreeBSD = 14.0 (1400066), stripped > $ readelf -e = /usr/local/poudriere/jails/pfSense_plus-devel-main_armv7/usr/bin/id >=20 > File: = /usr/local/poudriere/jails/pfSense_plus-devel-main_armv7/usr/bin/id > ELF Header: > Magic: 7f 45 4c 46 01 01 01 09 00 00 00 00 00 00 00 00 > Class: ELF32 > Data: 2's complement, little endian > Version: 1 (current) > OS/ABI: FreeBSD > ABI Version: 0 > Type: EXEC (Executable file) > Machine: ARM > Version: 0x1 > Entry point address: 0x20ef0 > Start of program headers: 52 (bytes into file) > Start of section headers: 8912 (bytes into file) > Flags: 0x5000400, Version5 EABI, VFP > Size of this header: 52 (bytes) > Size of program headers: 32 (bytes) > Number of program headers: 11 > Size of section headers: 40 (bytes) > Number of section headers: 28 > Section header string table index: 27 >=20 > Elf file type is EXEC (Executable file) > Entry point 0x20ef0 > There are 11 program headers, starting at offset 52 >=20 > Program Headers: > Type Offset VirtAddr PhysAddr FileSiz MemSiz Flg = Align > PHDR 0x000034 0x00010034 0x00010034 0x00160 0x00160 R = 0x4 > INTERP 0x000194 0x00010194 0x00010194 0x00015 0x00015 R = 0x1 > [Requesting program interpreter: /libexec/ld-elf.so.1] > LOAD 0x000000 0x00010000 0x00010000 0x00ccc 0x00ccc R = 0x10000 > LOAD 0x000ccc 0x00020ccc 0x00020ccc 0x01294 0x01294 R E = 0x10000 > LOAD 0x001f60 0x00031f60 0x00031f60 0x000c8 0x000c8 RW = 0x10000 > LOAD 0x002028 0x00042028 0x00042028 0x000a0 0x00138 RW = 0x10000 > DYNAMIC 0x001f68 0x00031f68 0x00031f68 0x000c0 0x000c0 RW = 0x4 > GNU_RELRO 0x001f60 0x00031f60 0x00031f60 0x000c8 0x000c8 R = 0x1 > GNU_STACK 0x000000 0x00000000 0x00000000 0x00000 0x00000 RW 0 > NOTE 0x0001ac 0x000101ac 0x000101ac 0x00064 0x00064 R = 0x4 > ARM_EXIDX 0x00090c 0x0001090c 0x0001090c 0x00068 0x00068 R = 0x4 >=20 > Section to Segment mapping: > Segment Sections... > 00 > 01 .interp > 02 .interp .note.tag .dynsym .gnu.version .gnu.version_r = .gnu.hash .hash .dynstr .rel.dyn .ARM.exidx .rel.plt .rodata .ARM.extab > 03 .text .init .fini .plt > 04 .jcr .init_array .dynamic > 05 .data .got.plt .bss > 06 .dynamic > 07 .jcr .init_array .dynamic > 08 > 09 .note.tag > 10 .ARM.exidx > There are 28 section headers, starting at offset 0x22d0: >=20 > Section Headers: > [Nr] Name Type Addr Off Size ES Flg = Lk Inf Al > [ 0] NULL 00000000 000000 000000 00 = 0 0 0 > [ 1] .interp PROGBITS 00010194 000194 000015 00 A = 0 0 1 > [ 2] .note.tag NOTE 000101ac 0001ac 000064 00 A = 0 0 4 > [ 3] .dynsym DYNSYM 00010210 000210 0002c0 10 A = 8 1 4 > [ 4] .gnu.version SUNW_versym 000104d0 0004d0 000058 02 A = 3 0 2 > [ 5] .gnu.version_r SUNW_verneed 00010528 000528 000050 00 A = 8 2 4 > [ 6] .gnu.hash GNU_HASH 00010578 000578 000030 00 A = 3 0 4 > [ 7] .hash HASH 000105a8 0005a8 000168 04 A = 3 0 4 > [ 8] .dynstr STRTAB 00010710 000710 0001e3 00 A = 0 0 1 > [ 9] .rel.dyn REL 000108f4 0008f4 000018 08 AI = 3 0 4 > [10] .ARM.exidx ARM_EXIDX 0001090c 00090c 000068 00 A = 14 0 4 > [11] .rel.plt REL 00010974 000974 000120 08 AI = 3 22 4 > [12] .rodata PROGBITS 00010a94 000a94 00021f 01 AMS = 0 0 1 > [13] .ARM.extab PROGBITS 00010cb4 000cb4 000018 00 A = 0 0 4 > [14] .text PROGBITS 00020ccc 000ccc 000ff0 00 AX = 0 0 4 > [15] .init PROGBITS 00021cc0 001cc0 000014 00 AX = 0 0 16 > [16] .fini PROGBITS 00021ce0 001ce0 000014 00 AX = 0 0 16 > [17] .plt PROGBITS 00021d00 001d00 000260 00 AX = 0 0 16 > [18] .jcr PROGBITS 00031f60 001f60 000004 00 WA = 0 0 4 > [19] .init_array INIT_ARRAY 00031f64 001f64 000004 00 WA = 0 0 4 > [20] .dynamic DYNAMIC 00031f68 001f68 0000c0 08 WA = 8 0 4 > [21] .data PROGBITS 00042028 002028 000004 00 WA = 0 0 4 > [22] .got.plt PROGBITS 0004202c 00202c 00009c 00 WA = 0 0 4 > [23] .bss NOBITS 00042100 0020c8 000060 00 WA = 0 0 64 > [24] .comment PROGBITS 00000000 0020c8 0000b6 01 MS = 0 0 1 > [25] .ARM.attributes ARM_ATTRIBUTES 00000000 00217e 000049 00 = 0 0 1 > [26] .gnu_debuglink PROGBITS 00000000 0021c7 000010 00 = 0 0 1 > [27] .shstrtab STRTAB 00000000 0021d7 0000f7 00 = 0 0 1 > Key to Flags: > W (write), A (alloc), X (execute), M (merge), S (strings) > I (info), L (link order), G (group), x (unknown) > O (extra OS processing required) o (OS specific), p (processor = specific) >=20 > It=E2=80=99s not quite clear to me how this is supposed to work (now). = On amd64 there=E2=80=99s a separate /libexec/ld-elf32.so.1, which we = don=E2=80=99t have on aarch64. Is it supposed to be built? >=20 > It=E2=80=99s broken on ab9293239c7d and e03b7883e97c at the very = least. [I'm ignoring qemu, which I do not use. The below is from a Cortex-A72 aarch64 context that can execute Cortex-A7 armv7 code as well. Have you been using qemu?] Historically I've only been able to execute armv7 FreeBSD code on aarch64 FreeBSD via using the likes of, say, chroot'ing into an installed armv7 world in a directory tree that I created for such. (I manually split some liong-lineouptut for readabilty.) # uname -apKU FreeBSD CA72_16Gp_ZFS 14.0-CURRENT FreeBSD 14.0-CURRENT #63 main-n258610-ba7319e9091b-dirty: Fri Oct 14 14:29:14 PDT 2022 = root@CA72_16Gp_ZFS:/usr/obj/BUILDs/main-CA72-nodbg-clang/usr/main-src/arm6= 4.aarch64/sys/GENERIC-NODBG-CA72 arm64 aarch64 1400072 1400072 # chroot /usr/obj/DESTDIRs/main-CA7-chroot/ # uname -apKU FreeBSD CA72_16Gp_ZFS 14.0-CURRENT FreeBSD 14.0-CURRENT #63 main-n258610-ba7319e9091b-dirty: Fri Oct 14 14:29:14 PDT 2022 = root@CA72_16Gp_ZFS:/usr/obj/BUILDs/main-CA72-nodbg-clang/usr/main-src/arm6= 4.aarch64/sys/GENERIC-NODBG-CA72 arm armv7 1400072 1400072 # which date /bin/date # file /bin/date /bin/date: ELF 32-bit LSB executable, ARM, EABI5 version 1 (FreeBSD), = dynamically linked, interpreter /libexec/ld-elf.so.1, FreeBSD-style, for = FreeBSD 14.0 (1400072), not stripped # date Tue Oct 18 17:28:13 UTC 2022 Direct execution attempts from an aarch64 world without such a chroot (or equivalent for the purpose) to a armv7 world have always produced the likes of: # /usr/obj/DESTDIRs/main-CA7-chroot/bin/date ELF interpreter /libexec/ld-elf.so.1 not found, error 8 Abort trap =3D=3D=3D Mark Millard marklmi at yahoo.com