From nobody Mon Mar 14 09:19:58 2022 X-Original-To: freebsd-xen@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id A68D51A0E3D9 for ; Mon, 14 Mar 2022 09:20:13 +0000 (UTC) (envelope-from prvs=0654505ba=roger.pau@citrix.com) Received: from esa1.hc3370-68.iphmx.com (esa1.hc3370-68.iphmx.com [216.71.145.142]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mx1.hc3370-68.iphmx.com", Issuer "HydrantID Server CA O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4KH9x83W8Mz4Svn for ; Mon, 14 Mar 2022 09:20:12 +0000 (UTC) (envelope-from prvs=0654505ba=roger.pau@citrix.com) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1647249612; h=date:from:to:cc:subject:message-id:references: in-reply-to:mime-version; bh=zCfDSlM/lVdduegzct8WpWHliePoj+iSDH1o/dhDOpw=; b=EjZ6S9ZsgkIlhr//RjrEp0zpU1MGng7SZ7yX4btLmwL2w22i44d0VPg/ nGtSN0cge5wQ1VPbUKaDcANdKaddQ2qhAyrC7QBGCPaehs0geXUBTTcv2 cpg+fjLYejrqLjn5aD9wkzjopkn8LZr9n4fkxqDW1voGnY4k7p1wJ9nNL 4=; X-SBRS: 5.1 X-MesageID: 66556240 X-Ironport-Server: esa1.hc3370-68.iphmx.com X-Remote-IP: 162.221.156.83 X-Policy: $RELAYED IronPort-Data: A9a23:n/3eJKuiYlRQU0LxxH1fkqhvi+fnVFReMUV32f8akzHdYApBsoF/q tZmKT3SOP2MYmSmf9x0a9ywpE4BsMWEn4BiSldsrnw3FSgU+JbJXdiXEBz9bniYRiHhoOOLz Cm8hv3odp1coqr0/0/1WlTZhSAgk/nOHNIQMcacUsxLbVYMpBwJ1FQyw4bVvqYy2YLjW1jU5 YuryyHiEATNNwBcYzp8B52r8HuDjNyq0N/PlgVjDRzjlAa2e0g9VPrzF4noR5fLatA88tqBb /TC1NmEElbxpH/BPD8HfoHTKSXmSpaKVeSHZ+E/t6KK2nCurQRquko32WZ1he66RFxlkvgoo Oihu6BcRi8kYLPQlM0XXyVXLGYkIqhg5I2aO2Sw5Jn7I03uKxMAwt1rBUAye4YZ5vx2ESdF8 vlwxDIlN07ZwbjsmfTiF7cq1p9LwMrDZevzvll6yj7UF7A+SI3rSKTW/95Imjw3g6iiGN6AO ZdEN2AzNHwsZTV0an0OOZs1xt3xqXagVmV49m/LlIsOtj27IAtZj+G2bYu9lsaxbc9Inlyfu UrA8W7yCxVcM8aQoRKL6H+hjPTD2DP2XqocDrC1/OVmxlqJyQQ7GhAaSEOgsOKRhUu3WtYZI EsRkhfCtoBrqhbtFIOkGUTl/jjU5XbwRua8DcU3th+RxoPe7zqWOS9eUTtndO57rtM5EGlCO kCyo/vlAjlmsbuwQH2b96uJoT7aBRX5PVPudgdfE1JbvoCLTJUby0uWE409SPLdYsjdQ2mY/ tyckMQpa1z/Z+Yv3r7zw13IiinESnPhHl9svVW/so5IA2pEiG+Zi26AtAKzARVodt/xory9U J8swZT20Qz2JcvR/BFhuc1UdF1T296LMSfHnXlkFIQ7+jKm9haLJN4Mv2gvex0xbZldIVcFh XM/XisLuPdu0IaCN/crM+pd9ex3pUQfKTgVfq+NNYcfCnSAXASG4DtvdSatM5PFyyARfVUEE c7DK66EVC9CYYw+lWbeb7pNgNcDm3FlrUuOFM+T8vhS+efHDJJjYexeawXmgyFQxP7snTg5B P4EbprUkUoDCLOiCsQVmKZKRW03wbEALcmeg+Rcd/KZIxogH2ckCvTLxqgmdZAjlKNQ/tokN FnmMqOE4DITXUH6FDg= IronPort-HdrOrdr: A9a23:530SqqCGpztEDZvlHelW55DYdb4zR+YMi2TDt3oddfWaSKylfq GV7ZAmPHrP4gr5N0tOpTntAse9qBDnhPtICOsqTNSftWDd0QPFEGgL1+DfKlbbak/DH4BmtJ uJc8JFeaDN5VoRt7eH3OFveexQv+Vu88qT9JnjJ28Gd3AMV0n5hT0JcTpyFCdNNW97LKt8Lr WwzOxdqQGtfHwGB/7LfEXsD4D41qT2fIuNW29/OyIa X-IronPort-AV: E=Sophos;i="5.90,180,1643691600"; d="scan'208";a="66556240" ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=RoLXOWMe1aJGR32YeGXFjfoeunwIo0Fqk8Fn5A3WxlpjvlVl2fidGgHKnVMY9nHh0pTb2T7szP6Twm9vCpNnLPoSW1nvPZraYB3fgMfufqAAYpDsMGATHhlzX0m8JcNeTa7BcgyrEl4wb/RlBy9SpHdaPbRETt6liZ841y/D/w6aqW9Ya1J5KUfSS3YU+I4aABh5B/l17+thP9eVb8hPIXCk1Qu2BDNOUoO4ZcLfWduln8MW7HheX/Uu/Ilok3fDJGdYkE/o9ZItCvE44Dq108vWWPmd1a3CFx0QbJOVYyNhYeZFpaWuQMgYOwYRYl+rTJGnNTp/JS220M9dd6CUAg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=u8uz4XcyU/QnoJnJVkQ8jtu3W3GsKGxiOLPgDdsngU8=; b=kqGXG7K85x6cz77U2EzzedXuj/XdvlsiXRED7Dn+03xktsyGbXLbof3MFV154ehFZ0kKzjSLPmI1HE8avMqfUuv5/6ywJ0FS1u1rYQbQ9rOwFNUrW6uOCn0RB+SXVLBGoN+4qg7Y1L7jO4VvpTNyKBHDqeoMeCc4eiyRaCo5zEuYszqQ4fRN7AOgyalFl0/MCyxEnNiyY8ApaMN0Y9ikzZGcrbcZ6cwCCodbFvBka5eK7FljFFKISrXi38rDnNpERlzu7WSEBbFp7KwPTifDFVgMIl3+YEJEhd0iRIJt+w+BRq6r9Tsk5FWf9oRXwaWvCyew0M4ZHoNBn+McSGr4mA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com; dkim=pass header.d=citrix.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=u8uz4XcyU/QnoJnJVkQ8jtu3W3GsKGxiOLPgDdsngU8=; b=SVsCCxkbUx1EufrqHSw2Rbj2L8rbJJXIlygphZm+t8StC15GY0IFrYh/1t/uHQ5OykYRMFzvEJzJFJbsWAciOGEWaYwJTDKObgygpu7galPiT6W74yfE7IsraIajSKscIc5ALZK10DXAnMJ01rZ8gXsWafR9rqkwIZ0pA09Bc4M= Date: Mon, 14 Mar 2022 10:19:58 +0100 From: Roger Pau =?utf-8?B?TW9ubsOp?= To: Ze Dupsys CC: , Subject: Re: ZFS + FreeBSD XEN dom0 panic Message-ID: References: <202203011540.221FeR4f028103@nfbcal.org> <3d4691a7-c4b3-1c91-9eaa-7af071561bb6@gmail.com> <5dfdecd5-f94d-29b4-791e-0adde5405cf5@gmail.com> Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: X-ClientProxiedBy: FR3P281CA0024.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:1c::15) To DS7PR03MB5608.namprd03.prod.outlook.com (2603:10b6:5:2c9::18) List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-xen List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-xen@freebsd.org X-BeenThere: freebsd-xen@freebsd.org MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 1f63ba07-7697-4815-50ee-08da059bd157 X-MS-TrafficTypeDiagnostic: MWHPR03MB2493:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: MQD7ObEWl+nTGvUQGWU7J3UBiG2Ggp9KV803phLXYKoTAVkchwI22PcfQT/Biw/5O/MpdRTKpFsMl2blKdLOCs0VQ10DNyUWWTQJzPIPGqW95Hv99Qz4EsixsquHRsg5EXr/XYCJIMkP7jZGunde9MY/I7YDNEhLd73dEBMarPT/nha716NEUkrS2epWI2bZhvJYkVrocqzXZfbVHVWfyLrxK5vOyklChRqSTPxH5yxG8VtbfOczv1tigp+/+eRSsndOBsM5y1e41rVjV/zTGk3xDcxign5BIVWLHIWgsJT5234l5DaagD0fpzpo6vhNop4EyfdO/Q548IX/CTU8ENzLR/DWOw3hmKvvXZckp3J83EFB9z50N0MdWboHo/Ug4/eEbj9Jp/6m267o4EEwjDkpEAHb1RluSiZTICPJBxIDfvES3nry3lecnipDhXVA1dNFF4CF8VCXR0J3YdARaNG9EKMVAj4H/+UEsZ8iQfMxbIJBw9040WBfQMWMITq/b0t8vheFxM3YjkfLomRuukHo0VWNF3lpEppmSXGGLJsl/AzhDcEDH+Y49sKzoJUMkYMNcHNBZSV1i0VONuLo8AQKzbaTbPxJId1nZtYOoEB6blxvpVV4xd/1NH7bWdHn3KojDad2qF9GeaKDsL9YjA== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR03MB5608.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(7916004)(4636009)(366004)(4326008)(5660300002)(38100700002)(66556008)(508600001)(82960400001)(66476007)(6486002)(66946007)(8676002)(83380400001)(33716001)(2906002)(316002)(6916009)(8936002)(6506007)(186003)(26005)(86362001)(85182001)(6666004)(6512007)(9686003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?YnZFVmRQZlJWWkZZT2E4ZUc4V2d1ZVV6VnMySkZqbVRIS2NPMExra1JVL1Y3?= =?utf-8?B?dVJHTXhVTDZDc3RpdHZ4MDNUSGx0M1VLdGZhdTJXY3M0aTQ2MGR6U3h0TjBP?= =?utf-8?B?WGoxQnIzb2s2SExsMlM1K0lDc0JySytRWFIvdzdOelkwZ0JJNmlMQVcxMjNN?= =?utf-8?B?WTZKenpJd0RBQURVM0NCTUJSSTRLaENTbGlOUGpjdE1DVGczTTJHUGdyaGxM?= =?utf-8?B?OVFWeTB4Qzh3Wm9NWU1iNmhwSTUydDdDbk1QWkdpeGVWTllLTVNpQjBZZFBm?= =?utf-8?B?c1Y2OXgxdXFoVmJYM0RZSmNoNCtmajBkMmxZWlBMR1VDN2RTTEcrM0Jaampr?= =?utf-8?B?OG42d1oyYS9EaHNMOUNGdDljWmliY0tGQjhTUHA5eDMxMWNsMWorWWZaalJm?= =?utf-8?B?NmVFTU5NTTk3RVU2azk4UkdCZWZEc3Q2UUZUS0dUcWFKQ0ZxTGFLQnUvVnho?= =?utf-8?B?NjBHZUNCOUFNcFdYaWZuRnJHUkVXNnFrSlF4UUsxN1dIdk5ZTDIwNlZERHg1?= =?utf-8?B?dkMzaGdYekROcGw0WXI1Mi9CMkpiYWo2WWQ2dG81S09ZKzYrakFzNXRGV1dx?= =?utf-8?B?U2NsaW5kT1VPKzNsbDcvSGlSRTIrQ3EwQkJPc1dha1FVUElpbGsvQ1pXOVBH?= =?utf-8?B?NktoS0RJOTVnLzhmUWIydlNuSVRPOXlnbTVHMi9STDh4TjBldEJTUVpkZVRO?= =?utf-8?B?L090T0ZrNW5EdGdZWk1nS3RuMkMyV3hUYUhrWW1nRU5zUDloaHNiK01oZHFx?= =?utf-8?B?K09OTjFvOWZJRXMwdkF4RzZUSVhZZ2V3dXNMaEZzYjNOR1ZtN2JyOFliNS83?= =?utf-8?B?ZGdxN2NlMUxpTG5uaFZsb0pVZ1FFWmpLTmJoeFRUYUZyQVFKSTE4MS92dk53?= =?utf-8?B?UjJuQVV5cXROUmMwYmFRZmJVVUpiMXhZMDlDeTRnNDl0VEttTURvV2tZUHdX?= =?utf-8?B?QVptS1VMZlkyMWJkZ0FVd21Ga01aMFJTNWkzbEVBNFhxa2ovVDdwRW9NQ1c1?= =?utf-8?B?N2ZUb2d0Y2RSQXEvRDBoLzNjRTNzUldsdmJmZEJIVW5uNWRGUlNUbjZtYmdE?= =?utf-8?B?eHpvUWtyZlhiZ0lFMFBNempCZFpHUDVGWndtRE1zdWlZMFFpa2pkbHpla1V3?= =?utf-8?B?STVCYnd3USsxTTcwUE1uZlhjWm5Sczc5UmlWMnBENWhyTEp3MTZKTU4vd21E?= =?utf-8?B?MlN0V0dKeG5DUnV4UFcrdHVjUzYrZUtnQTJDNmpKUC9adWJMTjhBREhzcnBx?= =?utf-8?B?WHlGNUJnYUVhVFMrazZiSXhjYjFXUmNhMFk5RTdsTXhUQ1NrVDdNazUrQmRL?= =?utf-8?B?b2grNUtyMjJlSStiUFFMUVB3b2RCdGhZQUNmNENBOFhwUEZqUDJlY3VpWDY4?= =?utf-8?B?RU13VXFYL0J0MksxdnppT1JQN3kvZDRFc3VkRWFPTnpUZUtaTzVBWG9lV3pn?= =?utf-8?B?MnltWmszdEFCRzdqc0pqRUhFUHlMNkxNZDM3NXZpNldKWHUydktMeWxHWWp3?= =?utf-8?B?cDZ2VXo3ZVRJWXFJNDFRYngrWnFrZnZ5ZjI0V0hQTDlXbEovLy9iOGNoaVJQ?= =?utf-8?B?Y1B0cTdhbXBVUEpCWUVSTUtzcVB5UnQzMFRSaURvdkhBQU81Nks3WHVYb0hE?= =?utf-8?B?Y1hUa0JydmQzcnZqaVplTzY4MFJBRmZ2QjNQelc1V0hvYlExUTV3VzNwTS9q?= =?utf-8?B?SS9LR0JjSURrWSt0YnVZRGhnMVFiV2IzaS8ybFJOUVJ6bG5aY3d3K3Rqblo4?= =?utf-8?B?eXJjWmNCdkhVTmhFUWtJZ2VaWU5keTZwYXIramIzclROR1piTmNJd0VzNG9Q?= =?utf-8?B?cSt5T3k0OFFDR3NrSCtiY3h3OHAwY25tbm4zZ2N2RWhiVWZ1a2dIbStkcUx4?= =?utf-8?B?dDEyK0Jkd2o5bkVEYUp4d2tQdXZ6YW5ySjFxbWthZG9EWDRmSTF4eGdOTHlH?= =?utf-8?B?aWk4RVY1dU92ZkU0SW5UbGZJUWlpeW1VOGRQRFg0d1l3OXFJUU1TS2x6NkdY?= =?utf-8?B?OTZuQkNmOVJRPT0=?= X-MS-Exchange-CrossTenant-Network-Message-Id: 1f63ba07-7697-4815-50ee-08da059bd157 X-MS-Exchange-CrossTenant-AuthSource: DS7PR03MB5608.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Mar 2022 09:20:01.8607 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: Khs4nGc2zgyX5N0l2aEgrT3q0vrts8eo7l2/sDRxKKonwz5XVw6sTRz7HFNMudUuAihb0UKVjLdVHLQloDsIeQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR03MB2493 X-OriginatorOrg: citrix.com X-Rspamd-Queue-Id: 4KH9x83W8Mz4Svn X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=citrix.com header.s=securemail header.b=EjZ6S9Zs; dkim=pass header.d=citrix.onmicrosoft.com header.s=selector2-citrix-onmicrosoft-com header.b=SVsCCxkb; arc=pass ("microsoft.com:s=arcselector9901:i=1"); dmarc=pass (policy=reject) header.from=citrix.com; spf=pass (mx1.freebsd.org: domain of "prvs=0654505ba=roger.pau@citrix.com" designates 216.71.145.142 as permitted sender) smtp.mailfrom="prvs=0654505ba=roger.pau@citrix.com" X-Spamd-Result: default: False [-5.64 / 15.00]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+exists:216.71.145.142.spf.hc3370-68.iphmx.com]; RCVD_DKIM_ARC_DNSWL_MED(-0.50)[]; DKIM_TRACE(0.00)[citrix.com:+,citrix.onmicrosoft.com:+]; RCVD_IN_DNSWL_MED(-0.20)[216.71.145.142:from]; DMARC_POLICY_ALLOW(-0.50)[citrix.com,reject]; NEURAL_HAM_SHORT(-0.99)[-0.991]; FREEMAIL_TO(0.00)[gmail.com]; RCVD_COUNT_ZERO(0.00)[0]; MIME_TRACE(0.00)[0:+]; R_MIXED_CHARSET(0.56)[subject]; ASN(0.00)[asn:16417, ipnet:216.71.145.0/24, country:US]; FROM_NEQ_ENVFROM(0.00)[roger.pau@citrix.com,prvs=0654505ba=roger.pau@citrix.com]; ARC_ALLOW(-1.00)[microsoft.com:s=arcselector9901:i=1]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[citrix.com:s=securemail,citrix.onmicrosoft.com:s=selector2-citrix-onmicrosoft-com]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; FORGED_SENDER_VERP_SRS(0.00)[]; DWL_DNSWL_LOW(-1.00)[citrix.com:dkim]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MLMMJ_DEST(0.00)[freebsd-xen]; MID_RHS_NOT_FQDN(0.50)[] X-ThisMailContainsUnwantedMimeParts: N On Mon, Mar 14, 2022 at 10:06:58AM +0200, Ze Dupsys wrote: > I'd like to share more analysis on given problem. I do not know if this > somehow helps or not, but i have noticed that across all my saved serial log > outputs, panic messages follow after some of these lines. > > .. > (XEN) HVM d34v0 save: TSC_ADJUST > (XEN) HVM d34v0 save: CPU_MSR > (XEN) HVM34 restore: CPU 0 > xnb(xnb_detach:1330): > xnb(xnb_detach:1339): > .. => panic > > > Most of panics are like this > .. > (XEN) HVM d26v0 save: TSC_ADJUST > (XEN) HVM d26v0 save: CPU_MSR > (XEN) HVM26 restore: CPU 0 > .. => panic > > .. > (XEN) HVM d42v0 save: TSC_ADJUST > (XEN) HVM d42v0 save: CPU_MSR > (XEN) HVM42 restore: CPU 0 > xnb(xnb_detach:1330): > xnb(xnb_detach:1339): > xnb(xnb_detach:1330): > xnb(xnb_detach:1339): > .. => panic > > > This one i think had different stressing conditions than other's, but i > don't remember > .. > (XEN) HVM d660v0 save: CPU_MSR > (XEN) HVM660 restore: CPU 0 > (XEN) d659v0: upcall vector 93 > spin lock 0xffffffff81eaa780 (sched lock 1) held by 0xfffff8020152d000 (tid > 100434) too long > timeout stopping cpus > panic: spin lock held too long That one seems to be a watchdog panic, albeit it's quite likely cause by an out of memory condition. > .. => panic > > > For serial output in middle when there are no crashes i have noticed that > there are at least 2 different execution paths. > > For most VM's boot flow continues with serial lines like these: > .. > (XEN) HVM1 restore: CPU 0 > xnb(xnb_probe:1123): Claiming device 0, xnb > xnb(xnb_attach:1267): Attaching to backend/vif/1/0 > xnb(xnb_frontend_changed:1391): frontend_state=Initialising, > xnb_state=InitWait > (d1) HVM Loader > .. > > For some though, there are lines like these, but they still boot, it just > seemed that these lines might be a possible continuation for "unsuccessful > panic". > .. > (XEN) HVM3 restore: CPU 0 > xnb(xnb_detach:1330): > xnb(xnb_detach:1339): > xnb(xnb_detach:1330): > xnb(xnb_detach:1339): > xnb(xnb_probe:1123): Claiming device 0, xnb > xnb(xnb_attach:1267): Attaching to backend/vif/3/0 > xnb(xnb_frontend_changed:1391): frontend_state=Initialising, > xnb_state=InitWait > (d3) HVM Loader > .. > > Why those lines starting "xnb(xnb_detach:1330):" do not have any message? > Could it be that there is a bad pointer to message buffer that can not be > printed? And then sometimes panic happens because access goes out of allowed > memory region? Some messages in netback are just "\n", likely leftovers from debug. Can you try to stress the system again but this time with guests not having any network interfaces? (so that netback doesn't get used in dom0). Then if you could rebuild the FreeBSD dom0 kernel with the above patch we might be able to get a bit more of info about blkback shutdown. Thanks, Roger. ---8<--- diff --git a/sys/dev/xen/blkback/blkback.c b/sys/dev/xen/blkback/blkback.c index 792933402c93..84ebb9068881 100644 --- a/sys/dev/xen/blkback/blkback.c +++ b/sys/dev/xen/blkback/blkback.c @@ -125,7 +125,7 @@ __FBSDID("$FreeBSD$"); /** * \brief Define to enable rudimentary request logging to the console. */ -#undef XBB_DEBUG +#define XBB_DEBUG 1 /*---------------------------------- Macros ----------------------------------*/ /**