From nobody Tue Oct 15 05:15:32 2024 X-Original-To: freebsd-virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4XSMkW4HhJz5YKBF for ; Tue, 15 Oct 2024 05:15:47 +0000 (UTC) (envelope-from shamaz.mazum@gmail.com) Received: from mail-lf1-x134.google.com (mail-lf1-x134.google.com [IPv6:2a00:1450:4864:20::134]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4XSMkV5RSMz4jMn; Tue, 15 Oct 2024 05:15:46 +0000 (UTC) (envelope-from shamaz.mazum@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-lf1-x134.google.com with SMTP id 2adb3069b0e04-53a007743e7so169784e87.1; Mon, 14 Oct 2024 22:15:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1728969345; x=1729574145; darn=freebsd.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=4Y9rl7zgdw70ybhpilllPsqgzyakyFdwnn0BRXZkM1c=; b=KK+WpZC3SfCzVnVe+5LS9kuqqO8seAxM4c8IxodwcU66dBgsNbEQrbYZn3SVTngl2P skyV1E2cCUO2GHZ8eCyY7Wry8O5ik/0UdhLHvoqnOEqLcPI5VGZgGhhgvz5WYH/bFmBf lzxA0GS6InNMVr8/IHsVTH73I17lccItsSdZQuPLwreha4M3C9jLuOYcd4f43NyeCXpR bOcD45KaLgPx1SigHjBkl6TuQzkw398RH1vAUbeEDjCHCsFtJBInxgiav2Ff/uDw5QeU FWeYqOUvRUdPOO4Raq2yenj32XTraOZHu1dHoBZjYu0EU4QHks3CpMF3CileX/nEiTQB /ADQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728969345; x=1729574145; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=4Y9rl7zgdw70ybhpilllPsqgzyakyFdwnn0BRXZkM1c=; b=Aeggu/kzaMYajuLevV8p8XM4VOEa/epwkv9/deu24nVzV6OBL1bCcFMVvnGjs4U2MA dCiToK0ezSev2RKamV0OkqbamEnd3IdCYEiliKy6U43aU3Mc2JQLA3r0rcVnhrf+Y65b EyKAQSLVp1mhlP0R8qRvy5VGNzqTfU9Mpo5lDapsChZqKkSv8KfHeHenw8CyHc1YGGAG Um1/imowXvffdNtr9gteM2TJiGsV3Zb4GJSSeA0UYWN4xLIGJImidkXAX0tHtVIfyI4h lUQq1fyvqxZQm1X/zgKqhPizOMUCZ0QdSi34xBAlShFG2ufJ+K1WhqvP59DXXi6wmah+ edxg== X-Gm-Message-State: AOJu0YxV+R/fWRA2e9Lw6029mnYWhfs/hcWXWODU0R9pWrCDjjPpOpY8 aj5NQfAE7jpgtdS+YlyZ6afcpiEShGbn7sbtOU/CHzQic40Mr6Q755CJ607rN5snjimnLs2F48o 8H7vjnApHi1dJrDv8t9u+C/WPSzLkTeAbt4E= X-Google-Smtp-Source: AGHT+IGx7V3gs1ufm2/citaRI/dI3EgL082dJkuEESHjddMjegigI5ayBpjPhKSI/4IQ4E47jPac1SZI6/kS/2AHP9A= X-Received: by 2002:a05:6512:2811:b0:539:f4ab:5638 with SMTP id 2adb3069b0e04-539f4ab57c9mr2595338e87.60.1728969344072; Mon, 14 Oct 2024 22:15:44 -0700 (PDT) List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: freebsd-virtualization@freebsd.org Sender: owner-freebsd-virtualization@FreeBSD.org MIME-Version: 1.0 References: <17f4077d-647d-4848-9d6f-97f9886ef636@freebsd.org> <8b249b64-d041-4f12-b6cb-fdb528837f22@freebsd.org> <106b8500-a0ef-4095-af20-8c0f110ea739@freebsd.org> In-Reply-To: <106b8500-a0ef-4095-af20-8c0f110ea739@freebsd.org> From: Vasily Postnicov Date: Tue, 15 Oct 2024 05:15:32 +0000 Message-ID: Subject: Re: Running Mezzano in bhyve To: Peter Grehan Cc: freebsd-virtualization@freebsd.org Content-Type: multipart/alternative; boundary="00000000000094c86f06247d0be4" X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; TAGGED_FROM(0.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US] X-Rspamd-Queue-Id: 4XSMkV5RSMz4jMn X-Spamd-Bar: ---- --00000000000094c86f06247d0be4 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Regarding items 3) and 4): 3) Indeed, bhyve does not explicitly forbid writing to 0x3c. I meant the following. The interrupt line is set is pci_emul.c in bhyve: pci_set_cfgdata8(pi, PCIR_INTLINE, pirq_irq(ii->ii_pirq_pin)); Bhyve asserts interrupts with pci_irq_assert in amd64/pci_irq.c. We need this line: vm_isa_assert_irq(pi->pi_vmctx, pirq->reg & PIRQ_IRQ, pi->pi_lintr.ioapic_irq); pirq->reg & PIRQ_IRQ is literally the same as pirq_irq(ii->ii_pirq_pin). Now, if something (e.g. UEFI firmware, bootloader) writes to PCIR_INTLINE bhyve will still send interrupts with the number that was there before the write, while the OS will expect an interrupt with the new number. I treat this as a bug in bhyve (but it affects nobody, because newer OSes do not use the 8259 interrupt controller). 4) It's commenting the lock what makes an effect. I commented pci_generate_msi just in case because it's not needed for Mezzano, but runs protected by the mutex which is now gone. This is a backtrace and thread list when bhyve hangs up if the mutex is not commented out: (lldb) bt * thread #1, name =3D 'mevent', stop reason =3D signal SIGSTOP * frame #0: 0x000011adeaa37e2a libthr.so.3`_umtx_op_err at _umtx_op_err.S:38 frame #1: 0x000011adeaa479c0 libthr.so.3`__thr_umutex_lock(mtx=3D0x0000378ecca00888, id=3D101223) at thr_umtx.c:79:3 frame #2: 0x000011adeaa40eea libthr.so.3`mutex_lock_sleep(curthread=3D0x0000378ecc412000, m=3D0x0000378ecca00888, abstime=3D0x0000000000000000) at thr_mutex.c:699:9 frame #3: 0x000011adeaa3ed8f libthr.so.3`__Tthr_mutex_lock [inlined] mutex_lock_common(m=3D0x0000378ecca00888, abstime=3D0x0000000000000000, cvattach=3Dfalse, rb_onlist=3Dfalse) at thr_mutex.c:733:9 frame #4: 0x000011adeaa3ed4d libthr.so.3`__Tthr_mutex_lock(mutex=3D) at thr_mutex.c:752:9 frame #5: 0x000011a5c43e7b06 bhyve`vi_interrupt(vs=3D0x0000378ecc4b8000= , isr=3D'\x01', msix_idx=3D65535) at virtio.h:358:3 frame #6: 0x000011a5c43e6c86 bhyve`vq_interrupt(vs=3D0x0000378ecc4b8000= , vq=3D0x0000378ecc4b8038) at virtio.h:376:2 frame #7: 0x000011a5c43e6c44 bhyve`vq_endchains(vq=3D0x0000378ecc4b8038= , used_all_avail=3D0) at virtio.c:512:3 frame #8: 0x000011a5c43db348 bhyve`pci_vtnet_rx(sc=3D0x0000378ecc4b8000= ) at pci_virtio_net.c:271:4 frame #9: 0x000011a5c43dab53 bhyve`pci_vtnet_rx_callback(fd=3D6, type=3DEVF_READ, param=3D0x0000378ecc4b8000) at pci_virtio_net.c:403:2 frame #10: 0x000011a5c43bb9f8 bhyve`mevent_handle(kev=3D0x000011ade4451200, numev=3D1) at mevent.c:273:3 frame #11: 0x000011a5c43bb5d7 bhyve`mevent_dispatch at mevent.c:549:3 frame #12: 0x000011a5c43aed4b bhyve`main(argc=3D1, argv=3D0x000011ade4453418) at bhyverun.c:1052:2 frame #13: 0x000011adec6c1a6a libc.so.7`__libc_start1(argc=3D24, argv=3D0x000011ade4453360, env=3D0x000011ade4453428, cleanup=3D, mainX=3D(bhyve`main at bhyverun.c:694)) at libc_start1.c:157:7 frame #14: 0x000011a5c43a80cd bhyve`_start at crt1_s.S:83 (lldb) frame select 5 frame #5: 0x000011a5c43e7b06 bhyve`vi_interrupt(vs=3D0x0000378ecc4b8000, isr=3D'\x01', msix_idx=3D65535) at virtio.h:358:3 355 if (pci_msix_enabled(vs->vs_pi)) 356 pci_generate_msix(vs->vs_pi, msix_idx); 357 else { -> 358 VS_LOCK(vs); 359 vs->vs_isr |=3D isr; 360 pci_generate_msi(vs->vs_pi, 0); 361 #ifdef __amd64__ (lldb) thread list Process 3185 stopped * thread #1: tid =3D 101223, 0x000011adeaa37e2a libthr.so.3`_umtx_op_err at _umtx_op_err.S:38, name =3D 'mevent', stop reason =3D signal SIGSTOP thread #2: tid =3D 101868, 0x000011adeaa37e2c libthr.so.3`_umtx_op_err at _umtx_op_err.S:38, name =3D 'blk-3:0-0', stop reason =3D signal SIGSTOP thread #3: tid =3D 101869, 0x000011adeaa37e2c libthr.so.3`_umtx_op_err at _umtx_op_err.S:38, name =3D 'blk-3:0-1', stop reason =3D signal SIGSTOP thread #4: tid =3D 101870, 0x000011adeaa37e2c libthr.so.3`_umtx_op_err at _umtx_op_err.S:38, name =3D 'blk-3:0-2', stop reason =3D signal SIGSTOP thread #5: tid =3D 101871, 0x000011adeaa37e2c libthr.so.3`_umtx_op_err at _umtx_op_err.S:38, name =3D 'blk-3:0-3', stop reason =3D signal SIGSTOP thread #6: tid =3D 101872, 0x000011adeaa37e2c libthr.so.3`_umtx_op_err at _umtx_op_err.S:38, name =3D 'blk-3:0-4', stop reason =3D signal SIGSTOP thread #7: tid =3D 101873, 0x000011adeaa37e2c libthr.so.3`_umtx_op_err at _umtx_op_err.S:38, name =3D 'blk-3:0-5', stop reason =3D signal SIGSTOP thread #8: tid =3D 101874, 0x000011adeaa37e2c libthr.so.3`_umtx_op_err at _umtx_op_err.S:38, name =3D 'blk-3:0-6', stop reason =3D signal SIGSTOP thread #9: tid =3D 101875, 0x000011adeaa37e2c libthr.so.3`_umtx_op_err at _umtx_op_err.S:38, name =3D 'blk-3:0-7', stop reason =3D signal SIGSTOP thread #10: tid =3D 101876, 0x000011adeaa37e2c libthr.so.3`_umtx_op_err a= t _umtx_op_err.S:38, name =3D 'vtnet-5:0 tx', stop reason =3D signal SIGSTOP thread #11: tid =3D 101877, 0x000011adeaa37e2c libthr.so.3`_umtx_op_err a= t _umtx_op_err.S:38, name =3D 'hda-audio-output', stop reason =3D signal SIGS= TOP thread #12: tid =3D 101878, 0x000011adec7752ea libc.so.7`__sys_accept at _accept.S:4, name =3D 'rfb', stop reason =3D signal SIGSTOP thread #13: tid =3D 101879, 0x000011adec7726aa libc.so.7`__sys_ioctl at ioctl.S:4, name =3D 'vcpu 0', stop reason =3D signal SIGSTOP thread #14: tid =3D 101880, 0x000011adeaa37e2c libthr.so.3`_umtx_op_err a= t _umtx_op_err.S:38, name =3D 'vcpu 1', stop reason =3D signal SIGSTOP I think implementing IOAPIC in MEzzano is the best option indeed, but I have a little experience. I'll see what I can do. =D0=BF=D0=BD, 14 =D0=BE=D0=BA=D1=82. 2024=E2=80=AF=D0=B3. =D0=B2 22:52, Pet= er Grehan : > > 1) The problem with PIT. Can be solved as you proposed or by > > patching Mezzano. The bhyve patch would be the best option for that: > it's useful for > other older o/s's (DOS). > > > 2) Mezzano assumes that Intel AHCI controllers report no more than 6 > > ports. Can be solved by patching Mezzano or defining MAX_PORTS to be > > 6 in usr.sbin/bhyve/pci_ahci.c > > A Mezzano patch would be best for that. The bhyve man page has an > example with 8 disks attached so reducing the limit to 6 could hit > existing users. > > > 3) According to > > https://wiki.osdev.org/PCI#Message_Signaled_Interrupts > > , interrupt > > line config register must be RW. Bhyve does not support writing to > > it. I do not know a correct fix, this [1] workaround helps, however. > > Bhyve does support writing to that - your patch disables that, and my > guess is that when Mezzano sees this as zero (ie invalid) it then looks > for the irq line via the ACPI MADT (or other means). > > A quick look at Mezzano shows that it is still using the 8259 PIC for > interrupts. At the minimum it should be using the IOAPIC, or excessive > interrupt sharing will result, and possibly incorrect behaviour when > this happens. I think IOAPIC support could be added without a large > amount of effort, compared to e.g. MSI/MSI-x. > > > 4) Finally, I had a random deadlock in interrupt handling for the > > virtio-net device. Likewise, I do not know how to fix it correctly, > > but this [2] patch helped. > > Hmmm that seems strange: MSI interrupts aren't generated if they > haven't been setup/enabled by a guest. Commenting out the lock/unlock > code would seem to indicate a larger bug in play. Would it possible to > get some tracing on that segment of code e.g. a dtrace log ? > > > Do you have any ideas how to make proper patches for bhyve from > > these workarounds? > > The first one can be put in a phab diff, which I'll do. I think there's > still some more work involved for the others. > > later, > > Peter. > > > > --00000000000094c86f06247d0be4 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Regarding items 3) and 4):

3) Indeed, b= hyve does not explicitly forbid writing to 0x3c. I meant the following. The= interrupt line is set is pci_emul.c in bhyve:
=C2=A0pci_set_cfgd= ata8(pi, PCIR_INTLINE, pirq_irq(ii->ii_pirq_pin));
Bhyve a= sserts interrupts with pci_irq_assert in amd64/pci_irq.c. We need this line= : vm_isa_assert_irq(pi->pi_vmctx, pirq->reg & PIRQ_IRQ, pi->pi= _lintr.ioapic_irq);
pirq->reg & PIRQ_IRQ is literally the = same as pirq_irq(ii->ii_pirq_pin). Now, if something (e.g. UEFI firmware= , bootloader) writes to PCIR_INTLINE bhyve will still send interrupts with = the number that was there before the write, while the OS will expect an int= errupt with the new number. I treat this as a bug in bhyve (but it affects = nobody, because newer OSes do not use the 8259 interrupt controller).
=

4) It's commenting the lock what makes an effect. I= commented pci_generate_msi just in case because it's not needed for Me= zzano, but runs protected by the mutex which is now gone.
This is= a backtrace and thread list when bhyve hangs up if the mutex is not commen= ted out:

(lldb) bt
* thread #1, name =3D 'm= event', stop reason =3D signal SIGSTOP
=C2=A0 * frame #0: 0x000011ad= eaa37e2a libthr.so.3`_umtx_op_err at _umtx_op_err.S:38
=C2=A0 =C2=A0 fra= me #1: 0x000011adeaa479c0 libthr.so.3`__thr_umutex_lock(mtx=3D0x0000378ecca= 00888, id=3D101223) at thr_umtx.c:79:3
=C2=A0 =C2=A0 frame #2: 0x000011a= deaa40eea libthr.so.3`mutex_lock_sleep(curthread=3D0x0000378ecc412000, m=3D= 0x0000378ecca00888, abstime=3D0x0000000000000000) at thr_mutex.c:699:9
= =C2=A0 =C2=A0 frame #3: 0x000011adeaa3ed8f libthr.so.3`__Tthr_mutex_lock [i= nlined] mutex_lock_common(m=3D0x0000378ecca00888, abstime=3D0x0000000000000= 000, cvattach=3Dfalse, rb_onlist=3Dfalse) at thr_mutex.c:733:9
=C2=A0 = =C2=A0 frame #4: 0x000011adeaa3ed4d libthr.so.3`__Tthr_mutex_lock(mutex=3D&= lt;unavailable>) at thr_mutex.c:752:9
=C2=A0 =C2=A0 frame #5: 0x00001= 1a5c43e7b06 bhyve`vi_interrupt(vs=3D0x0000378ecc4b8000, isr=3D'\x01'= ;, msix_idx=3D65535) at virtio.h:358:3
=C2=A0 =C2=A0 frame #6: 0x000011a= 5c43e6c86 bhyve`vq_interrupt(vs=3D0x0000378ecc4b8000, vq=3D0x0000378ecc4b80= 38) at virtio.h:376:2
=C2=A0 =C2=A0 frame #7: 0x000011a5c43e6c44 bhyve`v= q_endchains(vq=3D0x0000378ecc4b8038, used_all_avail=3D0) at virtio.c:512:3<= br>=C2=A0 =C2=A0 frame #8: 0x000011a5c43db348 bhyve`pci_vtnet_rx(sc=3D0x000= 0378ecc4b8000) at pci_virtio_net.c:271:4
=C2=A0 =C2=A0 frame #9: 0x00001= 1a5c43dab53 bhyve`pci_vtnet_rx_callback(fd=3D6, type=3DEVF_READ, param=3D0x= 0000378ecc4b8000) at pci_virtio_net.c:403:2
=C2=A0 =C2=A0 frame #10: 0x0= 00011a5c43bb9f8 bhyve`mevent_handle(kev=3D0x000011ade4451200, numev=3D1) at= mevent.c:273:3
=C2=A0 =C2=A0 frame #11: 0x000011a5c43bb5d7 bhyve`mevent= _dispatch at mevent.c:549:3
=C2=A0 =C2=A0 frame #12: 0x000011a5c43aed4b = bhyve`main(argc=3D1, argv=3D0x000011ade4453418) at bhyverun.c:1052:2
=C2= =A0 =C2=A0 frame #13: 0x000011adec6c1a6a libc.so.7`__libc_start1(argc=3D24,= argv=3D0x000011ade4453360, env=3D0x000011ade4453428, cleanup=3D<unavail= able>, mainX=3D(bhyve`main at bhyverun.c:694)) at libc_start1.c:157:7=C2=A0 =C2=A0 frame #14: 0x000011a5c43a80cd bhyve`_start at crt1_s.S:83

(lldb) frame select 5
frame #5: 0x000011a5c43= e7b06 bhyve`vi_interrupt(vs=3D0x0000378ecc4b8000, isr=3D'\x01', msi= x_idx=3D65535) at virtio.h:358:3
=C2=A0 =C2=A0355 if (pci_msix_enabled= (vs->vs_pi))
=C2=A0 =C2=A0356 pci_generate_msix(vs->vs_pi, msix= _idx);
=C2=A0 =C2=A0357 else {
-> 358 VS_LOCK(vs);
=C2=A0 = =C2=A0359 vs->vs_isr |=3D isr;
=C2=A0 =C2=A0360 pci_generate_ms= i(vs->vs_pi, 0);
=C2=A0 =C2=A0361 #ifdef __amd64__
(lldb) thread list
Process 3185 stopped
* thread #1: tid = =3D 101223, 0x000011adeaa37e2a libthr.so.3`_umtx_op_err at _umtx_op_err.S:3= 8, name =3D 'mevent', stop reason =3D signal SIGSTOP
=C2=A0 thre= ad #2: tid =3D 101868, 0x000011adeaa37e2c libthr.so.3`_umtx_op_err at _umtx= _op_err.S:38, name =3D 'blk-3:0-0', stop reason =3D signal SIGSTOP<= br>=C2=A0 thread #3: tid =3D 101869, 0x000011adeaa37e2c libthr.so.3`_umtx_o= p_err at _umtx_op_err.S:38, name =3D 'blk-3:0-1', stop reason =3D s= ignal SIGSTOP
=C2=A0 thread #4: tid =3D 101870, 0x000011adeaa37e2c libth= r.so.3`_umtx_op_err at _umtx_op_err.S:38, name =3D 'blk-3:0-2', sto= p reason =3D signal SIGSTOP
=C2=A0 thread #5: tid =3D 101871, 0x000011ad= eaa37e2c libthr.so.3`_umtx_op_err at _umtx_op_err.S:38, name =3D 'blk-3= :0-3', stop reason =3D signal SIGSTOP
=C2=A0 thread #6: tid =3D 1018= 72, 0x000011adeaa37e2c libthr.so.3`_umtx_op_err at _umtx_op_err.S:38, name = =3D 'blk-3:0-4', stop reason =3D signal SIGSTOP
=C2=A0 thread #7= : tid =3D 101873, 0x000011adeaa37e2c libthr.so.3`_umtx_op_err at _umtx_op_e= rr.S:38, name =3D 'blk-3:0-5', stop reason =3D signal SIGSTOP
= =C2=A0 thread #8: tid =3D 101874, 0x000011adeaa37e2c libthr.so.3`_umtx_op_e= rr at _umtx_op_err.S:38, name =3D 'blk-3:0-6', stop reason =3D sign= al SIGSTOP
=C2=A0 thread #9: tid =3D 101875, 0x000011adeaa37e2c libthr.s= o.3`_umtx_op_err at _umtx_op_err.S:38, name =3D 'blk-3:0-7', stop r= eason =3D signal SIGSTOP
=C2=A0 thread #10: tid =3D 101876, 0x000011adea= a37e2c libthr.so.3`_umtx_op_err at _umtx_op_err.S:38, name =3D 'vtnet-5= :0 tx', stop reason =3D signal SIGSTOP
=C2=A0 thread #11: tid =3D 10= 1877, 0x000011adeaa37e2c libthr.so.3`_umtx_op_err at _umtx_op_err.S:38, nam= e =3D 'hda-audio-output', stop reason =3D signal SIGSTOP
=C2=A0 = thread #12: tid =3D 101878, 0x000011adec7752ea libc.so.7`__sys_accept at _a= ccept.S:4, name =3D 'rfb', stop reason =3D signal SIGSTOP
=C2=A0= thread #13: tid =3D 101879, 0x000011adec7726aa libc.so.7`__sys_ioctl at io= ctl.S:4, name =3D 'vcpu 0', stop reason =3D signal SIGSTOP
=C2= =A0 thread #14: tid =3D 101880, 0x000011adeaa37e2c libthr.so.3`_umtx_op_err= at _umtx_op_err.S:38, name =3D 'vcpu 1', stop reason =3D signal SI= GSTOP

I think implementing IOAPIC in MEzzano i= s the best option indeed, but I have a little experience. I'll see what= I can do.

=D0=BF=D0=BD, 14 =D0=BE=D0=BA=D1=82. 2024=E2=80=AF=D0=B3. = =D0=B2 22:52, Peter Grehan <grehan= @freebsd.org>:
> 1) The problem with PI= T. Can be solved as you proposed or by
> patching Mezzano. The bhyve patch would be the best option for that: i= t's useful for
other older o/s's (DOS).

> 2) Mezzano assumes that Intel AHCI controllers report no more than 6 <= br> > ports. Can be solved by patching Mezzano or defining MAX_PORTS to be <= br> > 6 in usr.sbin/bhyve/pci_ahci.c

=C2=A0 A Mezzano patch would be best for that. The bhyve man page has an example with 8 disks attached so reducing the limit to 6 could hit
existing users.

> 3) According to
> https://wiki.osdev.org/PCI#Message_Signal= ed_Interrupts
> <https://wiki.osdev.org/PCI#Message_Si= gnaled_Interrupts>, interrupt
> line config register must be RW. Bhyve does not support writing to > it. I do not know a correct fix, this [1] workaround helps, however.
=C2=A0 Bhyve does support writing to that - your patch disables that, and m= y
guess is that when Mezzano sees this as zero (ie invalid) it then looks
for the irq line via the ACPI MADT (or other means).

=C2=A0 A quick look at Mezzano shows that it is still using the 8259 PIC fo= r
interrupts. At the minimum it should be using the IOAPIC, or excessive
interrupt sharing will result, and possibly incorrect behaviour when
this happens. I think IOAPIC support could be added without a large
amount of effort, compared to e.g. MSI/MSI-x.

> 4) Finally, I had a random deadlock in interrupt handling for the
> virtio-net device. Likewise, I do not know how to fix it correctly, > but this [2] patch helped.

=C2=A0 Hmmm that seems strange: MSI interrupts aren't generated if they=
haven't been setup/enabled by a guest. Commenting out the lock/unlock code would seem to indicate a larger bug in play. Would it possible to
get some tracing on that segment of code e.g. a dtrace log ?

> Do you have any ideas how to make proper patches for bhyve from
> these workarounds?

=C2=A0 The first one can be put in a phab diff, which I'll do. I think = there's
still some more work involved for the others.

later,

Peter.



--00000000000094c86f06247d0be4--