ZFS corruption using zvols as backingstore for hvm VM's

[Brian Buhrow]

	hello Roger.  Sorry for the confusion.  I've been running NetBSD, or
trying to, in both PV and HVM modes.  Here is a mesage I sent to the
port-xen at netbsd.org mailing list a few days ago, detailing what I found
with running NetBSD in any mode under Xen as a domu.  Unfortunately, I
don't think I can provide a trace  of a crash to the xen server when the
HVM host misbehaves because I've had to move on in my work to get things
working and I found a working combination of  NetBSD and ZFS that let's me
proceed.  I've seen some discussion in the Xen documentation that suggests
one can run a Xen dom0 in a domu guest, allowing for sand box testing of
what you're describing without having to devote hardware to the issue.  Am
I correct in this and where might I find instructions on how to do it?

In any case, here's what I found.
-thanks
-Brian

From: Brian Buhrow <buhrow at nfbcal.org>
Date: Mon, 16 Nov 2020 14:59:48 -0800
Subject: Re: Panic: HYPERVISOR_mmu_update failed on old NetBSD-5.2 domU
Cc: port-xen at netbsd.org, buhrow at nfbcal.org

	hello Greg.  thanks for the pointers on Xen changes over the years and
the reasons for the changes.  After a lot more testing, thought and cvs
diffing, here is the state of the world, as I understand it, for the
record.

	NetBSD-5/i386 won't run under Xen-4.12 or newer, possibly older
versions of Xen, I didn't test, because it still has calls to
xpq_queue_flush and the like which were used in pre-xen3 days.  I believe,
through code inspection, but didn't test, that this is also true of
NetBSD-6/i386 and NetBSD-7/i386.  

	My next thought was to use NetBSD-5 in HVM mode, but it turns out this
is a show stopper for versions of NetBSD all the way through -current as of
May 24, 2020, see the link below, due to a cascade of bugs in the piixide(4)
emulator in Qemu and NetBSD-s inability to deal with partial success in
writing to disks on that chip set.  This failure is what causes the
original disk corruption I wrote about at the beginning of this thread.  As
an alternative, I tried enabling the SCSI LSI emulator in Qemu, but this
doesn't work because our esiop(4) driver doesn't play well with the
emulated LSI chips in Qemu.  No disk corruption, just timeouts when writing
to attached emulated sd(4) devices.  So, NetBSD as an HVM guest is pretty
much a bust!

	Fortunately, there is a solution.  NetBSD-5.2/amd64 DOMU's work fine
under Xen-4.12 and above.  So, I'll run the 64-bit amd-64 base, with a
32-bit pkgsrc set of packages to allow me to migrate my busy, working
machine, to newere hardware as a VM, thereby giving me the ability to build
a replacement installation for this machine and begin building a new
environment without interrupting service on the old one.  I have several
instances of 64-bit kernels runing with 64-bit base and 32-bit pkgsrc
packages running successfully, so, while it's not ideal, it works quite
well and gives me space to think about how to move to NetBSD-9 and beyond.


	Thanks again for the ideas and, hopefully, someone will find this
summary useful.
-Brian

http://mail-index.NetBSD.org/source-changes/2020/05/24/msg117668.html