QEMU instance with scsi devices for RISC-V ?

From: Dennis Clarke <dclarke_at_blastwave.org>
Date: Tue, 01 Feb 2022 12:38:43 UTC
I am going in tight little circles trying to get more than 8 disks
to be available to a RISC-V instance. However the virtio block device
type seems to be limited to three bits for the device id. That is
just a guess.

I am trying with virtio-scsi-pci type but the LiveCD option does not
see anything other than the installer disk1.iso cdrom image :

riscv@pluto$
riscv@pluto$ /usr/local/bin/qemu-system-riscv64 \
 > -machine virt -m 16384M -cpu sifive-u54 -smp 4 -nographic \
 > -bios /usr/local/share/opensbi/lp64/generic/firmware/fw_jump.elf \
 > -kernel /usr/local/share/u-boot/u-boot-qemu-riscv64/u-boot.bin \
 > -drive 
file=/opt/qemu/riscv/fbsd14/fbsd13_riscv64_20220127_2c449a4c5a3-252673-disc1.iso,readonly=on,media=cdrom,id=hd0 
\
 > -device virtio-blk-device,drive=hd0 \
 > -device virtio-scsi-pci,id=scsi0 \
 > -drive 
file=/opt/qemu/riscv/fbsd14/hack_dev/scsi_dev_0.img,format=qcow2,if=none,id=scsi0000 
\
 > -device scsi-hd,drive=scsi0000 \
 > -object rng-random,filename=/dev/urandom,id=rng0 \
 > -device virtio-rng-device,rng=rng0 \
 > -device virtio-net-device,netdev=usernet \
 > -netdev user,id=usernet,hostfwd=tcp::10000-:22

OpenSBI v0.9
    ____                    _____ ____ _____
   / __ \                  / ____|  _ \_   _|
  | |  | |_ __   ___ _ __ | (___ | |_) || |
  | |  | | '_ \ / _ \ '_ \ \___ \|  _ < | |
  | |__| | |_) |  __/ | | |____) | |_) || |_
   \____/| .__/ \___|_| |_|_____/|____/_____|
         | |
         |_|

Platform Name             : riscv-virtio,qemu
Platform Features         : timer,mfdeleg
Platform HART Count       : 4
Firmware Base             : 0x80000000
Firmware Size             : 124 KB
Runtime SBI Version       : 0.2

Domain0 Name              : root
Domain0 Boot HART         : 2
Domain0 HARTs             : 0*,1*,2*,3*
Domain0 Region00          : 0x0000000080000000-0x000000008001ffff ()
Domain0 Region01          : 0x0000000000000000-0xffffffffffffffff (R,W,X)
Domain0 Next Address      : 0x0000000080200000
Domain0 Next Arg1         : 0x0000000082200000
Domain0 Next Mode         : S-mode
Domain0 SysReset          : yes

Boot HART ID              : 2
Boot HART Domain          : root
Boot HART ISA             : rv64imafdcsu
Boot HART Features        : scounteren,mcounteren,time
Boot HART PMP Count       : 16
Boot HART PMP Granularity : 4
Boot HART PMP Address Bits: 54
Boot HART MHPM Count      : 0
Boot HART MHPM Count      : 0
Boot HART MIDELEG         : 0x0000000000000222
Boot HART MEDELEG         : 0x000000000000b109


U-Boot 2021.07 (Nov 05 2021 - 03:54:51 +0000)

CPU:   rv64imafdcsu
Model: riscv-virtio,qemu
DRAM:  16 GiB
Loading Environment from nowhere... OK
In:    uart@10000000
Consoles: EFI console
    ______               ____   _____ _____
   |  ____|             |  _ \ / ____|  __ \
   | |___ _ __ ___  ___ | |_) | (___ | |  | |
   |  ___| '__/ _ \/ _ \|  _ < \___ \| |  | |
   | |   | | |  __/  __/| |_) |____) | |__| |
   | |   | | |    |    ||     |      |      |
   |_|   |_|  \___|\___||____/|_____/|_____/      ``` 
      `
                                                 s` 
`.....---.......--.```   -/
  +---------- Welcome to FreeBSD -----------+    +o   .--`         /y:` 
      +.
  |                                         |     yo`:.            :o 
    `+-
  |  1. Boot Single user [Enter]            |      y/               -/` 
   -o/
  |  2. Boot Multi user                     |     .- 
::/sy+:.
  |  3. Escape to loader prompt             |     / 
`--  /
  |  4. Reboot                              |    `: 
      :`
  |  5. Cons: Serial                        |    `: 
      :`
  |                                         |     / 
      /
  |  Options:                               |     .- 
     -.
  |  6. Kernel: default/kernel (1 of 1)     |      -- 
    -.
  |  7. Boot Options                        |       `:`                  `:`
  |                                         |         .--             `--.
  |                                         |            .---.....----.
  +-----------------------------------------+

Loading kernel...
/boot/kernel/kernel text=0x5eec5c text=0x1701d4 data=0xf7660 
data=0xbc4+0x2741b4 syms=[0x8+0xcffd8+0x8+0xf1ab5]
Loading configured modules...
can't find '/etc/hostid'
can't find '/boot/entropy'
Using DTB provided by EFI at 0x87efb000.
Kernel entry at 0xf660002e...
Kernel args: (null)
---<<BOOT>>---
GDB: no debug ports present
KDB: debugger backends: ddb
KDB: current backend: ddb
Physical memory chunk(s):
   0x80000000 - 0x47fffffff, 16384 MB (4194304 pages)
Excluded memory regions:
   0x80000000 - 0x801fffff,     2 MB (    512 pages) NoAlloc NoDump
   0xf6600000 - 0xf7325fff,    13 MB (   3366 pages) NoAlloc
Found 4 CPUs in the device tree
Copyright (c) 1992-2022 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
         The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 14.0-CURRENT #0 main-n252673-2c449a4c5a3: Thu Jan 27 05:19:37 
UTC 2022
 
root@releng1.nyi.freebsd.org:/usr/obj/usr/src/riscv.riscv64/sys/GENERIC 
riscv
FreeBSD clang version 13.0.0 (git@github.com:llvm/llvm-project.git 
llvmorg-13.0.0-0-gd7b669b3a303)
WARNING: WITNESS option enabled, expect reduced performance.
VT: init without driver.
Preloaded elf kernel "/boot/kernel/kernel" at 0xffffffc000d01000.
Preloaded TSLOG data "TSLOG" at 0xffffffc000d0a310.
SBI: OpenSBI v0.9
SBI Specification Version: 0.2
CPU(0): Unknown Implementer Unknown Processor
real memory  = 17179869184 (16384 MB)
Physical memory chunk(s):
0x0000000080200000 - 0x00000000f65fffff, 1983905792 bytes (484352 pages)
0x00000000f7326000 - 0x000000046668afff, 14750732288 bytes (3601253 pages)
avail memory = 16681529344 (15908 MB)
No static device mappings.
Starting CPU 2 (hart 0)
Starting CPU 3 (hart 1)
Starting CPU 1 (hart 3)
FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs
random: no preloaded entropy cache
arc4random: WARNING: initial seeding bypassed the cryptographic random 
device because it was not yet seeded and the knob 
'bypass_before_seeding' was enabled.
VIMAGE (virtualized network stack) enabled
hostuuid: using 00000000-0000-0000-0000-000000000000
ULE: setup cpu 0
ULE: setup cpu 1
ULE: setup cpu 2
ULE: setup cpu 3
random: entropy device external interface
openfirm: <Open Firmware control device>
kbd0 at kbdmux0
mem: <memory>
crypto: <crypto core>
null: <full device, null device, zero device>
ofwbus0: <Open Firmware Device Tree>
simplebus0: <Flattened device tree simple bus> on ofwbus0
plic0: <RISC-V PLIC> mem 0xc000000-0xc20ffff irq 10,11,12,13,14,15,16,17 
on simplebus0
timer0: <RISC-V Timer>
Timecounter "RISC-V Timecounter" frequency 10000000 Hz quality 1000
Event timer "RISC-V Eventtimer" frequency 10000000 Hz quality 1000
riscv_syscon0: <RISC-V syscon> mem 0x100000-0x100fff on simplebus0
rcons0: <RISC-V console>
ofwbus0: <fw-cfg@10100000> mem 0x10100000-0x10100017 compat 
qemu,fw-cfg-mmio (no driver attached)
ofwbus0: <flash@20000000> mem 
0x20000000-0x21ffffff,0x22000000-0x23ffffff compat cfi-flash (no driver 
attached)
cpulist0: <Open Firmware CPU Group> on ofwbus0
cpu0: <Open Firmware CPU> on cpulist0
cpu0: missing 'clock-frequency' property
riscv64_cpu0: register <0>
cpu1: <Open Firmware CPU> on cpulist0
cpu1: missing 'clock-frequency' property
riscv64_cpu1: register <1>
cpu2: <Open Firmware CPU> on cpulist0
cpu2: missing 'clock-frequency' property
riscv64_cpu2: register <2>
cpu3: <Open Firmware CPU> on cpulist0
cpu3: missing 'clock-frequency' property
riscv64_cpu3: register <3>
goldfish_rtc0: <Goldfish RTC> mem 0x101000-0x101fff irq 0 on simplebus0
goldfish_rtc0: registered as a time-of-day clock, resolution 1.000000s
uart0: <16550 or compatible> mem 0x10000000-0x100000ff irq 1 on simplebus0
uart0: console (115200,n,8,1)
uart0: fast interrupt
uart0: PPS capture mode: DCD
syscon_power0: <Syscon poweroff> on simplebus0
syscon_power1: <Syscon reboot> on simplebus0
pcib0: <Generic PCI host controller> mem 0x30000000-0x3fffffff on simplebus0
pcib0: parsing FDT for ECAM0:
pcib0:  PCI addr: 0x0, CPU addr: 0x3000000, Size: 0x10000
pcib0:  PCI addr: 0x40000000, CPU addr: 0x40000000, Size: 0x40000000
pcib0:  PCI addr: 0x800000000, CPU addr: 0x800000000, Size: 0x400000000
pcib0:  PCI addr: 0x0, CPU addr: 0x0, Size: 0x0
pcib0:  PCI addr: 0x0, CPU addr: 0x0, Size: 0x0
pcib0:  PCI addr: 0x0, CPU addr: 0x0, Size: 0x0
pcib0:  PCI addr: 0x0, CPU addr: 0x0, Size: 0x0
pcib0:  PCI addr: 0x0, CPU addr: 0x0, Size: 0x0
pcib0:  PCI addr: 0x0, CPU addr: 0x0, Size: 0x0
pcib0:  PCI addr: 0x0, CPU addr: 0x0, Size: 0x0
pcib0:  PCI addr: 0x0, CPU addr: 0x0, Size: 0x0
pcib0:  PCI addr: 0x0, CPU addr: 0x0, Size: 0x0
pcib0:  PCI addr: 0x0, CPU addr: 0x0, Size: 0x0
pcib0:  PCI addr: 0x0, CPU addr: 0x0, Size: 0x0
pcib0:  PCI addr: 0x0, CPU addr: 0x0, Size: 0x0
pcib0:  PCI addr: 0x0, CPU addr: 0x0, Size: 0x0
pcib0: Bus is cache-coherent
pci0: <PCI bus> on pcib0
pci0: domain=0, physical bus=0
found-> vendor=0x1b36, dev=0x0008, revid=0x00
         domain=0, bus=0, slot=0, func=0
         class=06-00-00, hdrtype=0x00, mfdev=0
         cmdreg=0x0004, statreg=0x0000, cachelnsz=8 (dwords)
         lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
found-> vendor=0x1af4, dev=0x1004, revid=0x00
         domain=0, bus=0, slot=1, func=0
         class=01-00-00, hdrtype=0x00, mfdev=0
         cmdreg=0x0007, statreg=0x0010, cachelnsz=8 (dwords)
         lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
         intpin=a, irq=0
         MSI-X supports 7 messages in map 0x14
         map[10]: type I/O Port, range 32, base 0x1000, size  6, enabled
pcib0: rman_reserve_resource: start=0x1000, end=0x103f, count=0x40
         map[14]: type Memory, range 32, base 0x40000000, size 12, enabled
pcib0: rman_reserve_resource: start=0x40000000, end=0x40000fff, count=0x1000
         map[20]: type Prefetchable Memory, range 64, base 0x40004000, 
size 14, enabled
pcib0: rman_reserve_resource: start=0x40004000, end=0x40007fff, count=0x4000
virtio_pci0: <VirtIO PCI (legacy) SCSI adapter> port 0x1000-0x103f mem 
0x40000000-0x40000fff,0x40004000-0x40007fff irq 26 at device 1.0 on pci0
virtio_mmio0: <VirtIO MMIO adapter> mem 0x10008000-0x10008fff irq 2 on 
simplebus0
vtblk0: <VirtIO Block Adapter> on virtio_mmio0
virtio_mmio0: host features: 0x31006ef4 
<RingEventIdx,RingIndirectDesc,NotifyOnEmpty,WriteZeros,Discard,ConfigWCE,Topology,FlushCmd,SCSICmds,BlockSize,ReadOnly,DiskGeometry,MaxNumSegs>
virtio_mmio0: negotiated features: 0x10002e74 
<RingIndirectDesc,Discard,ConfigWCE,Topology,FlushCmd,BlockSize,ReadOnly,DiskGeometry,MaxNumSegs>
vtblk0: 744MB (1524468 512 byte sectors)
virtio_mmio1: <VirtIO MMIO adapter> mem 0x10007000-0x10007fff irq 3 on 
simplebus0
virtio_mmio2: <VirtIO MMIO adapter> mem 0x10006000-0x10006fff irq 4 on 
simplebus0
vtnet0: <VirtIO Networking Adapter> on virtio_mmio2
virtio_mmio2: host features: 0x39bf8064 
<RingEventIdx,RingIndirectDesc,AnyLayout,NotifyOnEmpty,CtrlMacAddr,GuestAnnounce,CtrlRxModeExtra,CtrlVLANFilter,CtrlRxMode,CtrlVq,Status,MrgRxBuf,TxGSO,MAC,CtrlRxOffloads>
virtio_mmio2: negotiated features: 0x308f8064 
<RingEventIdx,RingIndirectDesc,CtrlMacAddr,CtrlVLANFilter,CtrlRxMode,CtrlVq,Status,MrgRxBuf,TxGSO,MAC,CtrlRxOffloads>
vtnet0: bpf attached
vtnet0: Ethernet address: 52:54:00:12:34:56
simplebus0: <virtio_mmio@10005000> mem 0x10005000-0x10005fff irq 5 
compat virtio,mmio (no driver attached)
simplebus0: <virtio_mmio@10004000> mem 0x10004000-0x10004fff irq 6 
compat virtio,mmio (no driver attached)
simplebus0: <virtio_mmio@10003000> mem 0x10003000-0x10003fff irq 7 
compat virtio,mmio (no driver attached)
simplebus0: <virtio_mmio@10002000> mem 0x10002000-0x10002fff irq 8 
compat virtio,mmio (no driver attached)
simplebus0: <virtio_mmio@10001000> mem 0x10001000-0x10001fff irq 9 
compat virtio,mmio (no driver attached)
simplebus0: <clint@2000000> mem 0x2000000-0x200ffff irq 
18,19,20,21,22,23,24,25 compat sifive,clint0 (no driver attached)
crypto: assign cryptosoft0 driver id 0, flags 0x6000000
Device configuration finished.
procfs registered
Timecounters tick every 1.000 msec
vlan: initialized, using hash tables with chaining
lo0: bpf attached
IPsec: Initialized Security Association Processing.
tcp_init: net.inet.tcp.tcbhashsize auto tuned to 131072
GEOM: new disk vtbd0
usb_needs_explore_all: no devclass
Release APs
CPU(1): Unknown Implementer Unknown Processor
CPU(3): Unknown Implementer Unknown Processor
CPU(2): Unknown Implementer Unknown Processor
WARNING: WITNESS option enabled, expect reduced performance.
Trying to mount root from cd9660:/dev/iso9660/14_0_CURRENT_RISCV64_CD 
[ro]...
GEOM: vtbd0: the secondary GPT table is corrupt or invalid.
GEOM: vtbd0: using the primary only -- recovery suggested.
GEOM: iso9660/14_0_CURRENT_RISCV64_CD: the secondary GPT table is 
corrupt or invalid.
GEOM: iso9660/14_0_CURRENT_RISCV64_CD: using the primary only -- 
recovery suggested.
cd9660: RockRidge Extension
goldfish_rtc0: providing initial system time
start_init: trying /sbin/init
Enter full pathname of shell or RETURN for /bin/sh:
random: unblocking device.
root@:/ # ls /dev/
audit           devctl          kbdmux0         random          urandom
auditpipe       devctl2         klog            reroot          usbctl
bpf             devstat         kmem            stderr          vtbd0
bpf0            fd              mdctl           stdin           vtbd0p1
console         fido            mem             stdout          xpt0
consolectl      full            msdosfs         sysmouse        zero
ctty            geom.ctl        null            ttyu0
cuau0           gptid           openfirm        ttyu0.init
cuau0.init      iso9660         pci             ttyu0.lock
cuau0.lock      kbd0            pfil            ufssuspend
root@:/ # exit
No suitable dump device was found.
Fast boot: skipping disk checks.
Mounting local filesystems:.
Building /boot/kernel/linker.hints
kldxref: can't create /boot/lhint.LIGgk6: Read-only file system
mkdir: 
/tmp/.diskless.d562ac475bd86c80dd789c8e6a152b9f93ce8629685d9e13e6d3f8682559c4cf: 
Read-only file system
devmatch: Can't read linker hints file.
Setting up harvesting: 
[UMA],[FS_ATIME],SWI,INTERRUPT,NET_NG,[NET_ETHER],NET_TUN,MOUSE,KEYBOARD,ATTACH,CACHED
Feeding entropy: dd: /entropy: Read-only file system
dd: /boot/entropy: Read-only file system
.
ELF ldconfig path: /lib /usr/lib /usr/lib/compat
/etc/rc: WARNING: $hostname is not set -- see rc.conf(5).
lo0: link state changed to UP
Starting Network: lo0 vtnet0.
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
         options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
         inet6 ::1 prefixlen 128
         inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2
         inet 127.0.0.1 netmask 0xff000000
         groups: lo
         nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
vtnet0: flags=8822<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
         options=80028<VLAN_MTU,JUMBO_MTU,LINKSTATE>
         ether 52:54:00:12:34:56
         media: Ethernet autoselect (10Gbase-T <full-duplex>)
         status: active
         nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
Starting devd.
devmatch: Can't read linker hints file.
devmatch: Can't read linker hints file.
devmatch: Can't read linker hints file.
devmatch: Can't read linker hints file.
devmatch: Can't read linker hints file.
devmatch: Can't read linker hints file.
devmatch: Can't read linker hints file.
devmatch: Can't read linker hints file.
Starting Network: vtnet0.
vtnet0: flags=8822<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
         options=80028<VLAN_MTU,JUMBO_MTU,LINKSTATE>
         ether 52:54:00:12:34:56
         media: Ethernet autoselect (10Gbase-T <full-duplex>)
         status: active
         nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
devmatch: Can't read linker hints file.
devmatch: Can't read linker hints file.
devmatch: Can't read linker hints file.
devmatch: Can't read linker hints file.
devmatch: Can't read linker hints file.
  FreeBSD Installer
 
──────────────────────────────────────────────────────────────────────────────






                      ┌────────────Welcome──────────────┐
                      │ Welcome to FreeBSD! Would you   │
                      │ like to begin an installation   │
                      │ or use the live CD?             │
                      ├─────────────────────────────────┤
                      │ <Install> < Shell > <Live CD>   │
                      └─────────────────────────────────┘








Mounting late filesystems:.
Starting cron.
Starting background file system checks in 60 seconds.

Tue Feb  1 12:22:28 UTC 2022

FreeBSD/riscv (Amnesiac) (ttyu0)

login: root
Feb  1 12:22:31  login[705]: ROOT LOGIN (root) ON ttyu0
FreeBSD 14.0-CURRENT (GENERIC) #0 main-n252673-2c449a4c5a3: Thu Jan 27 
05:19:37 UTC 2022

Welcome to FreeBSD!

Release Notes, Errata: https://www.FreeBSD.org/releases/
Security Advisories:   https://www.FreeBSD.org/security/
FreeBSD Handbook:      https://www.FreeBSD.org/handbook/
FreeBSD FAQ:           https://www.FreeBSD.org/faq/
Questions List:        https://www.FreeBSD.org/lists/questions/
FreeBSD Forums:        https://forums.FreeBSD.org/

Documents installed with the system are in the /usr/local/share/doc/freebsd/
directory, or can be installed later with:  pkg install en-freebsd-doc
For other languages, replace "en" with a language code like de or fr.

Show the version of FreeBSD installed:  freebsd-version ; uname -a
Please include that output and any error messages when posting questions.
Introduction to manual pages:  man man
FreeBSD directory layout:      man hier

To change this login announcement, see motd(5).
root@:~ #


Looking at the verbose output from boot I do see a SCSI device listed
there :

virtio_pci0: <VirtIO PCI (legacy) SCSI adapter> port 0x1000-0x103f mem 
0x40000000-0x40000fff,0x40004000-0x40007fff irq 26 at device 1.0 on pci0

So the bus id seems to be pci0 but thus far any attempt I have made
simply gets me a LiveCD running with no available SCSI disks.

Is this just a QEMU rabbit hole maze of options problem?

Dennis