From nobody Wed Mar 02 17:05:22 2022 X-Original-To: freebsd-xen@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id DBEEC19F5476 for ; Wed, 2 Mar 2022 17:05:34 +0000 (UTC) (envelope-from buhrow@nfbcal.org) Received: from nfbcal.org (ns.NFBCAL.ORG [157.22.230.125]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "nfbcal.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4K80qd1S7cz4Vb9 for ; Wed, 2 Mar 2022 17:05:32 +0000 (UTC) (envelope-from buhrow@nfbcal.org) Received: from nfbcal.org (localhost [127.0.0.1]) by nfbcal.org (8.15.2/8.14.1-NFBNETBSD) with ESMTPS id 222H5Muv016118 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Wed, 2 Mar 2022 09:05:23 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.102.2 at lothlorien.nfbcal.org Received: (from buhrow@localhost) by nfbcal.org (8.15.2/8.12.11) id 222H5MVm026787; Wed, 2 Mar 2022 09:05:22 -0800 (PST) Message-Id: <202203021705.222H5MVm026787@nfbcal.org> From: Brian Buhrow Date: Wed, 2 Mar 2022 09:05:22 -0800 In-Reply-To: X-Mailer: Mail User's Shell (7.2.6 beta(4.pl1)+dynamic 20000103) To: Ze Dupsys , freebsd-xen@freebsd.org Subject: Re: ZFS + FreeBSD XEN dom0 panic Cc: buhrow@nfbcal.org X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (nfbcal.org [127.0.0.1]); Wed, 02 Mar 2022 09:05:23 -0800 (PST) X-Rspamd-Queue-Id: 4K80qd1S7cz4Vb9 X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of buhrow@nfbcal.org designates 157.22.230.125 as permitted sender) smtp.mailfrom=buhrow@nfbcal.org X-Spamd-Result: default: False [-3.30 / 15.00]; MID_RHS_MATCH_FROM(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; RCPT_COUNT_THREE(0.00)[3]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+a:ns.nfbcal.org]; FROM_HAS_DN(0.00)[]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[nfbcal.org]; NEURAL_HAM_LONG(-1.00)[-1.000]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_HAM_SHORT(-1.00)[-1.000]; MLMMJ_DEST(0.00)[freebsd-xen]; FREEMAIL_TO(0.00)[gmail.com,freebsd.org]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:7091, ipnet:157.22.0.0/16, country:US]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[] X-ThisMailContainsUnwantedMimeParts: N List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-xen List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-xen@freebsd.org X-BeenThere: freebsd-xen@freebsd.org hello. One difference between my systems and yurs, though I don't think that's the problem, is that I'm not running a firewall on the dom0 itself. The dom0 runs on a protected vlan with respect to the external network and the domu's are connected to bridges that are directly connected to the external network. I have one system where the customer wants the pfsense system runing, so pfsense runs as a domu on this system, connected to an internal "private" bridge and the public bridge, doing all the firewalling between them. In this way, the FreeBSD dom0 is only doing ZFS, simple ip routing and Xen management. If I had to wager a guess as to your trouble, it's that you don't have enough memory on your dom0. ZFS is a memory hog and I can't imagine getting away with anything less than 8G on the dom0 with FreeBSD-12 and ZFS. I'm using 8G for the dom0 on the system I'm writing from and it is quite stable, but, then again, I'm not doing as much with the dom0 as you are. I too am using zvols as disks for the domu's, but I've not been trying to make zfs snapshots from them. obvious question, but I'll ask it anyway, you're not trying to make snapshots of the zvols while the domu's on top of them are running, are you? I would imagine that would not give you good images, but I wouldn't expect it to panic the dom0 either. However, it wil stretch your meager memory resources even further. Have you been able to get a panic message or does the system just spontaneously reboot? If it just reboots, then, again, I think you are having a memory shortage. My suggestion is to try giving the dom0 8G of RAM and then for the domU's, use the balloon driver to oversubscribe the remaining memory for the domu's. Of course, the best course of action is to see if you can put more memory in this system; 16GB just isn't that much when you're trying to run Xen plus a few domu's, especially on top of ZFS. If yu can get a panic message or a crash dump, that would be helpful in figuring out more accurately what's going on. Another thought, since you were getting some crashes when running jails with xen, is to get memtest86 running on the raw machine and let it run for 3 or 4 days. If you don't get any memory errors, then I think you can be pretty sure it's not a hardware problem. If, however, you get any errors at all with that test, then I think it's a good bet you hav a hardware issue. -thanks -Brian