From nobody Fri May 19 13:10:15 2023 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4QN6dw6llSz4Bwp3 for ; Fri, 19 May 2023 13:10:28 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-ej1-x630.google.com (mail-ej1-x630.google.com [IPv6:2a00:1450:4864:20::630]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4QN6dw4m31z3Kvp for ; Fri, 19 May 2023 13:10:28 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-ej1-x630.google.com with SMTP id a640c23a62f3a-965ddb2093bso521108966b.2 for ; Fri, 19 May 2023 06:10:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20221208.gappssmtp.com; s=20221208; t=1684501827; x=1687093827; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=qvdG2ZloS7De4xUpa+CKt8GjX888atMJdRQot3DIL28=; b=cevaAAIgC1GGpfdgIxsmQ7GynFy1Iwi6HXpzB3axr/G+FHjiUxC6j/7UfwG3sWXaJe 1ZRVXgX2goV1WiPqcX0MZph5DrzCvFppJjtI+HDFow3jVZ/5BC+btEvsm2cNfirZ/ZKM Soc3WJtDTonLiSjWJ1CfHKPXWm5ZTPTJQyhAGZTSLVm6fkbbrNcuHIfrq++Ld5MwROHA l9dmEnXtwwX94jVYMmWbMt9Q3dpH2isPVL318LBrCPQf3AjjkP7aIL48a7CM6KjtgGKw Dh8M+poIZet6Acc84l52IZIHdvBBwZq+b9A0gkomewAZX1XLUenCPt9dhbe7IDGVeXNG 1GPA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684501827; x=1687093827; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=qvdG2ZloS7De4xUpa+CKt8GjX888atMJdRQot3DIL28=; b=FmU/dokaFwwHoHa7wQRkav+PDzqYDbxK7qHkg7RjAtVxgdF1ajFst253REyhbUvx01 WysHIXH/pEOibrDn6WG+56YzLRsnP7cjGMzVdYem+WdIaMqOLzRoaTCaSCQviMg+Cb5z eaqZ55PojPo4uaB6FvFC1OEJ6pnCmXnFzMfVQ3W/xmdRDJ2RwhHnt329HX4BF68u4ZpM DLEHvA+C2TBTfYzgHnDhw/mL8RmdYYgV+evRpiNjylgKu1wWMhSQFDpx0r99hVk/P7ZG A9CfN6xTV9LSGIWbXDAPRLhiVgtzaWVxuESUPhj/RPXBTQTu7c1Qa09ifahTU/yD51Zp PYiA== X-Gm-Message-State: AC+VfDyFdoc4Nu1Cr8PRRBf//nzCRtJtdHG8nc2ppKjJe3SkqS9Y0F32 fWVyzpd9DndWdP5T5UJD3ApusF5R8OXkSQcoKh/thiimTiOEW/PoJ2Y= X-Google-Smtp-Source: ACHHUZ7QVnplneUzpVBnRqN4XIzCRSHt+sk/7gbjsXispH3ZgVnvNyy7qKUAyIXhUTj8jEKAlOUSJxAesWzKwkH+dxw= X-Received: by 2002:a17:907:a01:b0:966:265d:edc7 with SMTP id bb1-20020a1709070a0100b00966265dedc7mr1468967ejc.69.1684501827062; Fri, 19 May 2023 06:10:27 -0700 (PDT) List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org MIME-Version: 1.0 References: <202305190249.34J2n3hg029621@office.dignus.com> In-Reply-To: From: Warner Losh Date: Fri, 19 May 2023 07:10:15 -0600 Message-ID: Subject: Re: some QEMU success in running armv7 To: Mario Marietto Cc: Thomas David Rivers , freebsd-arm@freebsd.org Content-Type: multipart/alternative; boundary="000000000000069e3005fc0ba550" X-Rspamd-Queue-Id: 4QN6dw4m31z3Kvp X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N --000000000000069e3005fc0ba550 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable You'd never use kvm for armv7 when running on amd64 anyway... You can't run the code directly, but have to use the emulation stream. Warner On Fri, May 19, 2023 at 1:45=E2=80=AFAM Mario Marietto wrote: > Nice review. infortunately kvm does not work ? and a qemu vm without kvm > is very slow and useless. > > Il ven 19 mag 2023, 04:49 Thomas David Rivers ha > scritto: > >> >> Just f.y.i. - here's the steps I was able to deduce today >> to get FreeBSD armv7 running under qemu... just in case >> someone goes looking for this... (I was doing this on a >> FreeBSD 12.3-RELEASE x86_64 system.) >> >> >> 1. Retrieve the armv7 image from: >> fetch >> https://download.freebsd.org/releases/arm/armv7/ISO-IMAGES/13.2/FreeBSD-= 13.2-RELEASE-arm-armv7-GENERICSD.img.xz >> and unzip it. This is an actual hard-drive image of a working >> system. >> >> 2. Assuming the qemu port has been installed - this command >> starts that system >> >> qemu-system-arm -M virt -m 512m -nographic \ >> -bios edk2-arm-code.fd \ >> -hda FreeBSD-13.2-RELEASE-arm-armv7-GENERICSD.img >> >> the existing terminal will be the console. There is no >> networking. >> >> There root user's password is 'root'. >> >> Unfortunately this doesn't provide networking in the guest. This >> compilation of QEMU also doesn't support the "-netdev user" mode >> of networking so I can't take advantage of that. (The FreeBSD 12.3 >> host is using QEMU emulator version 7.2.0 - from just the pkg >> install.) However, I found that using this option on qemu: >> >> -nic tap,ifname=3Dtap7,script=3Dno,downscript=3Dno >> >> got it to use the tap interface on the host to emulate the virtio >> device to the guest. >> >> I also found this discussion regarding alignment issues in the >> virtio driver in armv7: >> >> >> https://forums.freebsd.org/threads/kernel-panic-on-armv7-with-qemu.89016= /#post-610281 >> >> that resulted in PR 271288. Apparently it's because of newer versions >> of QEMU doing a better job at reporting unaligned memory accesses in the >> guest for the armv7 "ldm" instruction. >> >> When I use the tap interface, I did get the exact panic mentioned >> in the forum and the PR. >> >> I did find that specifying the rtl8139 device worked around the panic >> with the QEMU option: >> >> -nic tap,ifname=3Dtap7,script=3Dno,downscript=3Dno,model=3Drtl8139 >> >> by the way - tap7 happens to be a tap device I'd already configured >> on the host FreeBSD 12.3 system - if you're doing this yourself, >> it will likely need to be a different tap device, see this >> write-up for info about how to configure a tap + bridge on your >> FreeBSD host: >> >> http://bsdwiki.reedmedia.net/wiki/networking_qemu_virtual_bsd_systems.ht= ml >> >> So - it seems a newer version of the armv7 kernel with the patch >> applied will fix the virtio driver problem, until then, model=3Drtl8139 >> works around it and I have networking and everything! >> >> - Dave R. - >> >> -- >> rivers@dignus.com Work: (919) 676-0847 >> Get your mainframe programming tools at http://www.dignus.com >> >> --000000000000069e3005fc0ba550 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
You'd never use kvm for armv7 when running on amd= 64 anyway... You can't run
the code directly, but have to use= the emulation stream.

Warner

On Fri, May 19, 2023= at 1:45=E2=80=AFAM Mario Marietto <marietto2008@gmail.com> wrote:
Nice review. infortunately kv= m does not work ? and a qemu vm without kvm is very slow and useless.
=
Il ven= 19 mag 2023, 04:49 Thomas David Rivers <rivers@dignus.com> ha scritto:

Just f.y.i. - here's the steps I was able to deduce today
to get FreeBSD armv7 running under qemu... just in case
someone goes looking for this... (I was doing this on a
FreeBSD 12.3-RELEASE x86_64 system.)


1.=C2=A0 =C2=A0Retrieve the armv7 image from:
=C2=A0 =C2=A0 =C2=A0 =C2=A0 fetch https://download.fr= eebsd.org/releases/arm/armv7/ISO-IMAGES/13.2/FreeBSD-13.2-RELEASE-arm-armv7= -GENERICSD.img.xz
=C2=A0 =C2=A0 =C2=A0and unzip it.=C2=A0 =C2=A0This is an actual hard-drive = image of a working
=C2=A0 =C2=A0 =C2=A0system.

2.=C2=A0 Assuming the qemu port has been installed - this command
=C2=A0 =C2=A0 starts that system

=C2=A0 =C2=A0 qemu-system-arm -M virt -m 512m -nographic \
=C2=A0 =C2=A0 =C2=A0 =C2=A0-bios edk2-arm-code.fd \
=C2=A0 =C2=A0 =C2=A0 =C2=A0-hda FreeBSD-13.2-RELEASE-arm-armv7-GENERICSD.im= g

=C2=A0 =C2=A0 the existing terminal will be the console.=C2=A0 There is no<= br> =C2=A0 =C2=A0 networking.

=C2=A0 =C2=A0 There root user's password is 'root'.

Unfortunately this doesn't provide networking in the guest.=C2=A0 This<= br> compilation of QEMU also doesn't support the "-netdev user" m= ode
of networking so I can't take advantage of that.=C2=A0 (The FreeBSD 12.= 3
host is using QEMU emulator version 7.2.0 - from just the pkg
install.)=C2=A0 However, I found that using this option on qemu:

=C2=A0 =C2=A0 -nic tap,ifname=3Dtap7,script=3Dno,downscript=3Dno

got it to use the tap interface on the host to emulate the virtio
device to the guest.

I also found this discussion regarding alignment issues in the
virtio driver in armv7:

=C2=A0 https://forums.freebsd.org/threads/kernel-panic-on-armv7-with-qemu.89016= /#post-610281

that resulted in PR 271288.=C2=A0 =C2=A0Apparently it's because of newe= r versions
of QEMU doing a better job at reporting unaligned memory accesses in the guest for the armv7 "ldm" instruction.

When I use the tap interface, I did get the exact panic mentioned
in the forum and the PR.

I did find that specifying the rtl8139 device worked around the panic
with the QEMU option:

=C2=A0 -nic tap,ifname=3Dtap7,script=3Dno,downscript=3Dno,model=3Drtl8139
by the way - tap7 happens to be a tap device I'd already configured
on the host FreeBSD 12.3 system - if you're doing this yourself,
it will likely need to be a different tap device, see this
write-up for info about how to configure a tap + bridge on your
FreeBSD host:
=C2=A0 =C2=A0ht= tp://bsdwiki.reedmedia.net/wiki/networking_qemu_virtual_bsd_systems.html

So - it seems a newer version of the armv7 kernel with the patch
applied will fix the virtio driver problem, until then, model=3Drtl8139
works around it and I have networking and everything!

=C2=A0 =C2=A0 =C2=A0 =C2=A0 - Dave R. -

--
r= ivers@dignus.com=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 Work: (919) 676-0847
Get your mainframe programming tools at http://www.dignus.com
--000000000000069e3005fc0ba550--