From nobody Fri Apr 19 12:44:06 2024 X-Original-To: xen@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VLZ8X1Vchz5H0WH for ; Fri, 19 Apr 2024 12:44:12 +0000 (UTC) (envelope-from roger.pau@cloud.com) Received: from mail-wm1-x336.google.com (mail-wm1-x336.google.com [IPv6:2a00:1450:4864:20::336]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4VLZ8W6fGRz447y for ; Fri, 19 Apr 2024 12:44:11 +0000 (UTC) (envelope-from roger.pau@cloud.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-wm1-x336.google.com with SMTP id 5b1f17b1804b1-418c979ddd2so14124295e9.3 for ; Fri, 19 Apr 2024 05:44:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1713530649; x=1714135449; darn=freebsd.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=Kt649tNTjw19InQd4+U98VX2Lv95WSulJkzWCM7Fe8s=; b=qCIOlDdV6EHo5+wNCCIIerpSAyM1l8wdWrrY7eYZhDHNWEvy8ojYF1uomB3T8yw3Kr X0o4L7IgYr1NB3U9SAfed1VZ7FMPXIDcLSS1xmdaplEtI+q72PnTgRjPvbYaRBDjdzCu lr1j+ygzRuglJPVwcBToB3EzulX9V7PrijzZY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713530649; x=1714135449; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=Kt649tNTjw19InQd4+U98VX2Lv95WSulJkzWCM7Fe8s=; b=H029uj9uZy97G0awZq98AnsKLE2DmxlnzERYRAs24tL7kSDsn20ek6sBKhEdfaezeq rUqdXZ1XIGIbiFt18oUFpU3CLTvohXicZ5V/vtUNbBRulR1mW4VA2IFNypPVgiXcKuJ9 Ut2BCQ5FxRauniODpHWiWz9u/I0gxitWw0JuEasZAAX541nonjfK2VR7pEYsT8BZlpFW Mn0741Bo4HgmQB95nGqYCPANUfBNeATiVE5lJ/EC+U8YluEcGGImdNJw3IFeMSKfEblZ GBpdeaBg4Z9UvcdjUhQO3/dcltUgv9Hus/8jT1TXuz9TxTn6uB41NlzFCtI8hz56ALRk A6NA== X-Gm-Message-State: AOJu0YwkgxFNXTnhq/zF+BVpdhbGZTJrM34U+kcta2mkelZvjqF3qnHM 6Z0oykn4hp86ik2rNnkf0yoC2wT6bSD9AZYNi0ttD88dPMsY+rzMSjV7oX1Et9wzjNa4EY4duy1 h X-Google-Smtp-Source: AGHT+IHEChhHeQdziYBj3/jsTGIemBuOesZ+dRpx0on1MAtJ95MZ+tkxty0owATN60MWO/DfGfpNrg== X-Received: by 2002:a05:600c:5254:b0:418:f827:160e with SMTP id fc20-20020a05600c525400b00418f827160emr1755010wmb.9.1713530648933; Fri, 19 Apr 2024 05:44:08 -0700 (PDT) Received: from localhost ([85.31.135.62]) by smtp.gmail.com with ESMTPSA id s9-20020a05600c45c900b004180c6a26bdsm5873142wmo.1.2024.04.19.05.44.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 19 Apr 2024 05:44:08 -0700 (PDT) Date: Fri, 19 Apr 2024 14:44:06 +0200 From: Roger Pau =?utf-8?B?TW9ubsOp?= To: Brian Buhrow Cc: xen@freebsd.org Subject: Re: Freebsd-13.1 as domu cannot use multiple virtual disks Message-ID: References: <202404190559.43J5x9eZ015862@nfbcal.org> List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-xen List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: freebsd-xen@freebsd.org Sender: owner-freebsd-xen@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <202404190559.43J5x9eZ015862@nfbcal.org> X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US] X-Rspamd-Queue-Id: 4VLZ8W6fGRz447y On Thu, Apr 18, 2024 at 10:59:09PM -0700, Brian Buhrow wrote: > Hello. I'm running FreeBSD-13.1 as a domu on xen-4.16.0 and I find I cannot use multiple > virtual disks on the same virtual guest. > > > The problem appears to be related to the way the geom layer sorts the disks. Somehow it thinks > the first and second disks are related. > > Here is the configuration of the disks for the virtual machine: > > > disk = [ > 'file:/var/xen/nfbcal/fbsd_dev/disk0,0x1,rw', > 'file:/var/xen/nfbcal/fbsd_dev/disk1,0x2,rw' > ] The vdev syntax that you are using is not a valid one, see: https://xenbits.xen.org/docs/unstable/man/xen-vbd-interface.7.html That wants to be: disk = [ 'file:/var/xen/nfbcal/fbsd_dev/disk0,xvda,rw', 'file:/var/xen/nfbcal/fbsd_dev/disk1,xvdb,rw' ] Or hd{a,b}. I've tried using 0x1 and 0x2 as your example, and it results in some errors in geom. > Here's what the virtual guest reports when it boots: > > > xbd0: 51200MB at device/vbd/1 on xenbusb_front0 > xbd0: features: flush, write_barrier > xbd0: synchronize cache commands enabled. > xbd1: 100000MB at device/vbd/2 on xenbusb_front0 > xbd1: features: flush, write_barrier > xbd1: synchronize cache commands enabled. > Trying to mount root from ufs:/dev/gpt/root []... > sysctl_warn_reuse: can't re-use a leaf (kern.geom.disk.xbd0.led)! > sysctl_warn_reuse: can't re-use a leaf (kern.geom.disk.xbd0.flags)! > g_dev_taste: make_dev_p() failed (gp->name=xbd0, error=17) > > > Any ideas what I'm doing wrong? this must work, right? It does work for me when using xvd{a,b}: [...] xbd0: 6176MB at device/vbd/51712 on xenbusb_front0 xbd0: features: flush, write_barrier xbd0: synchronize cache commands enabled. xbd1: 6176MB at device/vbd/51728 on xenbusb_front0 xbd1: features: flush, write_barrier xbd1: synchronize cache commands enabled. Trying to mount root from ufs:/dev/xbd0p4 []... WARNING: WITNESS option enabled, expect reduced performance. random: unblocking device. Setting hostuuid: 116f6f55-b478-11ee-a8a1-48210b35a577. Setting hostid: 0xa6037d1b. Starting file system checks: /dev/xbd0p4: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/xbd0p4: clean, 331160 free (2664 frags, 41062 blocks, 0.2% fragmentation) /dev/gpt/efiesp: FILESYSTEM CLEAN; SKIPPING CHECKS Mounting local filesystems:. [...] Regards, Roger.