From nobody Tue Jan 11 10:18:41 2022 X-Original-To: xen@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id CE79C194150D for ; Tue, 11 Jan 2022 10:18:58 +0000 (UTC) (envelope-from roger.pau@citrix.com) Received: from esa6.hc3370-68.iphmx.com (esa6.hc3370-68.iphmx.com [216.71.155.175]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mx1.hc3370-68.iphmx.com", Issuer "HydrantID SSL CA G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4JY69Z2Zk7z4n6r; Tue, 11 Jan 2022 10:18:58 +0000 (UTC) (envelope-from roger.pau@citrix.com) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1641896338; h=date:from:to:cc:subject:message-id:references: in-reply-to:mime-version; bh=5c+8ZG9NgZymF9xLsr0x8Ts0lWCWEvo7fWE0UdUdf9E=; b=aVt9xCPEd6N/l1Qpt0VoyKrR7S5FbYeZflniKbGLs2I+jQ1uPF3RsjCt XSPANepDJ8DjCoIplWEz6BATDAlbAWqgdSpO/Oqc3XptG/4uai6IGqwu2 vEDlUSdep6TtIiqtI9rdwFpysK7whsMwlKumRUBsGKIdlANyozX6ullPa c=; IronPort-SDR: 3UVNYqMg6fj8/I6CVVf1lyT/MepLAHpgYIqQI/K1xYitjA+V4gfPYMhcbOigq+r+PWfItUjlun OwKQPKfM7siUW5vbScx37WcjCEEZUwvWDi+dzzg7QnLDOS8+0u1eGlQ/8FruMoETzqTibooQM7 lbvYpY0zMcaqJUUiBO80uQgTe60Rjc4vXYvs76Lb/tQHrEGhpTwqyJ3XCUsttL7du1jKLnRbfW Dw7gexJJMQfXCp8laxwVYAB8YTeY4s7C3nJ4x1WGHVy/LkRavykvVMT4TcfAe5q87iHk0BOn4H Ze0EwcACMPqBaen9myoZstek X-SBRS: 5.2 X-MesageID: 61646110 X-Ironport-Server: esa6.hc3370-68.iphmx.com X-Remote-IP: 162.221.156.83 X-Policy: $RELAYED IronPort-Data: A9a23:KhYmJ6DgwsvWgRVW/+Pkw5YqxClBgxIJ4kV8jS/XYbTApGlx0GZTz mtOW2/VOP7YYzf0ftlwaoi1p0NV6JHVm4IxQQY4rX1jcSlH+JHPbTi7wuYcHM8wwunrFh8PA xA2M4GYRCwMo/u1Si6FatANl1ElvU2zbue6WL6s1hxZH1c+En940Es7wobVv6Yz6TSHK1LV0 T/Ni5W31G+Ng1aY5UpNtspvADs21BjDkGtwUm4WPJinj3eH/5UhN7oNJLnZEpfNatI88thW5 Qr05OrREmvxp3/BAz4++1rxWhVirrX6ZWBihpfKMkQLb9crSiEai84G2PQghUh/tTWDrchcl ZJxurOIUggpPPGTovkYXEwNe81+FfUuFL7vJHG+tYqYzlHccmuqyPJrZK00FdRGoKAtWzgIr KFGbmBWBvyAr7veLLaTUO5ji95lNMD2FIgepmth3XfSCvNOrZXrHfyUuYEBhW9YasZmP/vSW uwbUxRUNC/RRwNFanI2Os4ztbL97pX4W2IB8w/EzUYt2EDDwQpZ3qLgPdDOdpqNX8o9tk+Gq UrI5Wn0CAoWctuFxlK4HmmE37GV23mhAcRLSeP+pqUCbECvKnI7Nh0cZ0OW/L6FhWGGQN1fK RBLwDcrov1nnKC0deXVUxq9qX+CmxcTXdtMDuE3gD2wJrroDxWxXTZdEGMYADAynIpvHGFxi AfV9z/8LWE36NWopWShGqB4RN9YEQwcNic8aCANVmPpCPGz8dho3nojojuOeZNZb+EZ+xmtm 1hmTwBk3t3/aPLnMY3hpjgrZBr29/D0ovYdvFm/Y45f0it3ZZS+e6uj4kXB4PBLIe6xFwfd5 ShYw5HPtLtTVflhcRBhps1XR9lFAN7fYVXhbaNHRcF9p1xBBVb+FWyv3N2ODBgwaZtVEdMYS ETSpRlQ9Pdu0IiCNsdKj3aKI51yl8DITI29PtiNN4YmSsUvKGevoX8/DWbNjzGFuBV9yskXZ MbEGftA+F5HU8xP1iStfe4B3NcDn2ZmrY8lbcqln03PPHv3TCP9dIrpx3PVP79pt/3V8VyFm zudXuPToyhivCTFSnC/2aYYLEwQLGh9ApbzqsdNcfWELBYgE2YkY8I9C5t4E2C8t6gKxOrO4 F+nXUpUlAj2iXHdcF3YYXF/crL/G514qCtjbyArOF+p3VklYJqus/hDJ8dmI+F/+bwx1+NwQ tkEZ96EXqZFRAPY9mlPdpL6toFjKkim3FrcIyq/bTEjVJd8XAiVqMT8dw7i+XBWXCq6vMczu ZO60QbfTcZRTghuFp+OOvmu00mwrT4Wn+crBxnEJdxaeUPN9ol2KnOu0q9rcp9UcRian2mUz QebBxsctNLhmY5t/Ymbn72AoqeoD/B6QhhQEV7E4OvkLiLd5Gein9NNCb7aYTDHWWro06y+f uEJnerkOfgKkVsW4YpxF7FnkfA369f1/uIIyw1lGDPAbkixC6MmKX6DhJEduqpIz75fmA23R kPQpYUKZeTXYJvoQAwLOQ4oTuWfzvVFyDDd4MM8LFj++CIqrqGMVl9fPkXUhSFQRFev3FjJH Qv1VBYq1jGC IronPort-HdrOrdr: A9a23:R1/ia6EmyBU3eyOHpLqFDJHXdLJyesId70hD6qkvc3Jom52j+P xGws526faVslYssHFJo6HnBEClewKgyXcV2/hqAV7GZmjbUQSTXeRfBOfZslnd8mjFh5JgPM RbAtlD4b/LfCBHZK/BiWHSebtQo6jkzEnrv5ak854Ed3AVV0gK1XYBNu/0KDwQeOEQbqBJa6 Z0q/A37AaISDAyVICWF3MFV+/Mq5nik4/nWwcPA1oC5BOVhT2lxbbmG1zAty1uHg9n8PMHyy zoggb57qKsv7WSzQLd7Xba69BzlMH6wtVOKcSQgow+KynqiCyveIN9Mofy9gwdkaWK0hIHgd PMqxAvM4Ba7G7QRHi8pV/X1wzpwF8Vmgnf4G7dpUGmjd3yRTo8BcYEr5leaAHl500pu8w5+L 5X3kqC3qAnQy/orWDY3ZzlRhtqnk27rT4JiugIlUFSVoMYdft4sZEfxkVIC50NdRiKp7zPKN MeTP002cwmMm9zNxvizytSKZ2XLzgO9y69Mwk/Upf/6UkSoJh7p3Fos/D30E1wsK7VcKM0lN gsBJ4Y4I2mfvVmHZ6VO91xM/dfKla9CC4kY1jiaWgOKsk8SgfwQtjMkfII2N0= X-IronPort-AV: E=Sophos;i="5.88,279,1635220800"; d="scan'208";a="61646110" ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=CSmJQQZr13S+e9/AnS4lp8pCKAMmqhl2fJtwe4zJhp11OlDBfyToEy7KoQ+1wUhQzMVpwZutCOF5O63AQRjOCgtr8Hb/en8g9yFkXalCERcm+skNByS8/zK2pZCTcDnhr2f9NCj/8qbefRFaoIoJU//EpTT8hlc7hU5pOflDrOmqeOfta6IZ9oNqf34m05ndCbQ9nD2VY8HqIvWNd9YWgA3rhnkmEn+h/PabjNm7iOcmnnfDsv3cuoEP+jwtvczM6PgG4BEaUHlWd76NxiPNwwOv6xCasa6ZvsCktcmvqBLWRIuGaTiNuO/Rm5CYNtgAm7oKNFwdfAmZkC4TxsSC+Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=9uK0eYXxJkfFEEzso+oAWEhPTOkqx3YrVD2ErWNxQGw=; b=Uy4ZEOQG0s3fihErW6LkztH9NMrK4r8PfzXn9p80fcHRfpT93mqFbGfQftBwEsJL0cH47Xm7pC/3cY64RD65xHVZURx5odCKNJvzhYDCmo+tb1iBlXLaSvd0LerWowP71Y769yLyYDREy26YkqSrdldbmX6NikYiFTgKRXfEVKBnqKQz1++2M61RL1OUNneotJihQ/6aqPOxbzyRR55SmjEtB4F24TYOs5oB58iYYFedqXtIS3OlB6OVz4HiSA0YK8XqsbsBEFHTnok0aZ24ldJGlf5BM78w6XAUUQH+1tCFdf2dhzrYvx2laSLxE6d+irgmEpalR8pDptu+i5U2SQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com; dkim=pass header.d=citrix.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=9uK0eYXxJkfFEEzso+oAWEhPTOkqx3YrVD2ErWNxQGw=; b=Ymd5cQMrimcTut0qn2oLJ13TekSze3tPAjl42Z6ICe2PPRUwASJ+9TAtE5MyJrexRVVNAqO+7MgFicp2k05uqJOLKmedqYpMv3KRtlSXNqng6kB8P6+zrTvv2ZEuIpNQDbQui1ZaoZp2FWnwWKsA0JAgA2FkJQejlx6AHyOX4nY= Date: Tue, 11 Jan 2022 11:18:41 +0100 From: Roger Pau =?utf-8?B?TW9ubsOp?= To: Andriy Gapon CC: Subject: Re: VCPUOP_send_nmi returns -38 Message-ID: References: <3f9f173b-40b9-0180-404d-52fa56dde45f@FreeBSD.org> Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <3f9f173b-40b9-0180-404d-52fa56dde45f@FreeBSD.org> X-ClientProxiedBy: MR1P264CA0056.FRAP264.PROD.OUTLOOK.COM (2603:10a6:501:3e::20) To DS7PR03MB5608.namprd03.prod.outlook.com (2603:10b6:5:2c9::18) List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-xen List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-xen@freebsd.org X-BeenThere: freebsd-xen@freebsd.org MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: a40dd13e-4534-434e-ef82-08d9d4ebc0cb X-MS-TrafficTypeDiagnostic: DM5PR03MB3291:EE_ X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:8882; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 7OAbh3fVbyCOv7zKHSfyhaykNbboAuFvE+7PV+HbseBf/AKeVQLi9epWBlGIVC1mxCrtZAh75+X4GfIBnpmYx6krtMR3oiKzQXOEdZqcfni/QHy2S43lZ8f4s511Odir3gGoApq3mFxTYOVAVUS948risubb8YAHZvbXvq8PSltWQ6W3PIlANeuxse1wzYx4VN3WoMislxlkGJ+66ZV1rOm+xZ1k3KGsNBlUuEqlS1/DVDloAll2RP91S3hoymH1J+np3jPKl6r0gJSBaMKp0L41pj0ERJ/E1+5WxMy63zWqHUS8aIVbv0Makddq9NDbMd3+p8XVt/W/S57QcNCcAcbMqO5PXudvQyIVyiS5t+Hz+3zI3U8vkU2ZnHTPgtcG6cOiWFxl51oxMRj4LeRXqgRMgLYwVZ4212g4c+Qy92uiTk72IeW4eSfH2fOF5xveky95MuAWmz6v3c8bzDYK4dvxhiy9iW4Z/tQtZHPL5UZJZi43klJE1Vb5RAlM5Ts5PFr50khefvb/0SpDYcFyhHCzEf+oqHClwZdlYWlvYjrN0tFU6e++l0RWzwBpMu0aoaCE4nhuOXpiSsGM0BIfOpPAbT0t5vMeVaaSkFEhdKitg7H0gDedrgbkvzVgTgYn0SAsk/aGj9MHWyrVHpRa3Q== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR03MB5608.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(7916004)(4636009)(366004)(2906002)(38100700002)(8676002)(86362001)(66946007)(6666004)(8936002)(7116003)(26005)(5660300002)(4326008)(83380400001)(85182001)(9686003)(316002)(6512007)(33716001)(82960400001)(66556008)(450100002)(66476007)(186003)(6486002)(6506007)(6916009)(508600001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?alJWS2hOYmQ2ZlJ2ZmlzS2lQcFdFOVc3RTBwNzFrN2lWU1ZDYWVnWGdOZUxB?= =?utf-8?B?dkd6b29qS2RPSTYwemNUaGNlZXdvVm5oTVFSMHJXbGljaS9EZEFsMTZCS3l2?= =?utf-8?B?bysxZGxXSm1DVmtYTGZHU1VEZzJSUGhpdStOdTU2YTdrdDY5ZW5JRUg3czMy?= =?utf-8?B?b3dJMUp3dENDU0l6aml6c2VsTUJnVnFkSGRFU21wOHZwZmYzeGk2ZGV2NHQ1?= =?utf-8?B?NE9ON0ZtL2I3K0hzbjc4UnQzMXdEVlNOK3EvSndPOUlBQndMYjV3R2xWa1ZI?= =?utf-8?B?bXBIbHFoaGF1TXU1d3hDTXRpdWdqQzZvMzc4U1RkZHpiS0o4WmQ3UlZvdVpO?= =?utf-8?B?YXVGKzZYTVlFa3loNHRLeWFLNXhQb243QU52Z3RHVGlGSk4rZTMrYlppbVJH?= =?utf-8?B?cExCZ3ZRcWpmY2czK21TR2QxMEVIWGdQNjJOVHlURE9vUlJSS3Z3THVpMXZv?= =?utf-8?B?amg1WHBPbHBkNkxEWUNUNzBVY3NRNHQ3VlZXN1hTMHlVVW5DSTBQK1ZGY0pV?= =?utf-8?B?U2M4ZWtuM0VKK2hCRWZrU3JWUVRRVWJaUkxwVkZ4RUgwS0p5czhLcGJyR25P?= =?utf-8?B?c0hPbVpZTzNjbzM4RE9Ubm44YSs2aFpTTk1kVmpacEJjWXhCZzhGcUZmcU05?= =?utf-8?B?VzVMMWNVZXhMMFhqODQwYVBGamI1Nmp1aWE5VnlYYU9uTVdOeitpdWtjVnNx?= =?utf-8?B?MnpYQnlzMzVjVUxLUklJOUx5SlIxY1gwVXlOYkZZSnZuM0MwTWxsaFFia3BB?= =?utf-8?B?VDhCb2VoeFAvd2tYZ0VTRWJrTmlLZmovYnVndVRTS1kzaWhIckVTbzlVeXNa?= =?utf-8?B?STE2a1Q5TzAzSXlQSFRJMktNZ2NXL2lMWThuUnQ4MWIxVlR0THdDWUthM0pv?= =?utf-8?B?KzVHNHRySnJDSTIvT1FrQ0U3N1FWZ0cwTmRUTTNPSkV1STZKeXFLNVlscHVM?= =?utf-8?B?TEV4ZTlYSFU3RHRZR3Vra1hxbEFSek9OdWc3bS96dWZtcFBRNUpLR0pFQ1pB?= =?utf-8?B?WHJjWEloRHdUY1NSckNZVzRFd2NVSTBpTFF1S2RkNWp1MGh2TWx0ME9hQWdw?= =?utf-8?B?a21IcTVyRVBVd1ZhZUNXOGNCSWZVUmNyR3lEaXBUOXEzMENmQzFKNk4wWFhG?= =?utf-8?B?R1RrcHM1azlnK2h4aks5cXEwMitFdDY3b1pEY2srS2FoZ05BdkNSQWdwS0Ju?= =?utf-8?B?UWFiRTJvL2tBY3FIcFBmSDlKMmpDV0lqMll5bjVReW5lUGdhZ1k3YitneGNs?= =?utf-8?B?cmM3Nmkva0JnR3pLZzRrYlZ6SXd5WlVUMVJsVUxYZCtadEF5aEVFL05ORkU1?= =?utf-8?B?VDhuRlR3MUFaWmhYcTlWS1N1UGh6K1JoK3lac0VUUG5aejJidGQxb3phYW8w?= =?utf-8?B?YzJJOU9PSWRpbkdRazI4dUQxQnRFQU1OSmpDTy9oSDVLN3hocUZBY3I5NE5J?= =?utf-8?B?Sk5zQm5KTWpUVkcrdTRVYVlYMjliNDNGcXZmRlNPay9adXVRTVhsL0lNUHNJ?= =?utf-8?B?RlZ2K0NGNHhlVHB3bkVGaXVvZWQvb3BUbFZkaEgxMCtKRElJV2ZtYmsrTkYz?= =?utf-8?B?YnhlQkZ2S2VMN2cySzgvT3ZGbm1UNFdNN0xWZ2tKTjdQMFJsN3FLT25xdzhB?= =?utf-8?B?cENLcUxpZ3ptRnhFWndQbmd2NjdCRnpJUE5mUzI2VFppZTZpd1BnT3RvSGxv?= =?utf-8?B?SzF5VzFsQVF0TDhrcmVsTzBrL29sazQ1T3YwU2lVZzJEVU5pb2RYc0FXZ2xN?= =?utf-8?B?cVdPQnVLMjRrV0QycUk2QlZrN3BzaHZKdGd4VC9ib3JRc2pha0s3Ymg3NlRN?= =?utf-8?B?YXR0My9ENjVnQ3d4bzAzL2FvdGRNNzA4cjdVMjA4T2NqVmQ2V2RobnlpNzVP?= =?utf-8?B?dDNZOUo5S1pGRWtWcEgzTnBsT3lKcFJtQXJDUk9TSWF4UnlTMTg1WU1QNVZt?= =?utf-8?B?b2VCeGRFSlY1aFppTTJhZzRnZWxLZ0o0K2VGbGNGUlR5K3d5eTNQd2ZVVnVt?= =?utf-8?B?ajRZM0MvSDYxSCtNZTRZQ1VlZW9namxCTE1wOTJuQkw2TkNXYnpXaVRDMmQw?= =?utf-8?B?YVk0WDFPMlNSM21VZXMxY0ZhcmdmeGpDVUdRYysza1N2QUdNNzExeTRHOW1X?= =?utf-8?B?NnBCNTlBNkJrMW83eCtzUjBwL0tsUmRPd0pBeTBKb1cvdER5S0JLWFNwcTQw?= =?utf-8?Q?1FfKWUAL9PnpVAvFr5HzhkM=3D?= X-MS-Exchange-CrossTenant-Network-Message-Id: a40dd13e-4534-434e-ef82-08d9d4ebc0cb X-MS-Exchange-CrossTenant-AuthSource: DS7PR03MB5608.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Jan 2022 10:18:46.8203 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: OY/y/ZxZZPWkfOQzv5ZXzUeoOjs8xPQL8f+ZaoQnpnkxLpPQeQ0v9x5XkZztosvQ3a+udBmeElv047lOKZvXaA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR03MB3291 X-OriginatorOrg: citrix.com X-Rspamd-Queue-Id: 4JY69Z2Zk7z4n6r X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-ThisMailContainsUnwantedMimeParts: N On Tue, Jan 11, 2022 at 11:50:53AM +0200, Andriy Gapon wrote: > > Recently I got a report of crashes related to using procstat -k on one of > our systems. The system runs FreeBSD 12.2 on an AWS Xen-based instance (can > get more specifics about it later). > > It immediately reminded me of an older issue (Subject: Xen (HVM) and NMI) > where the root cause was that NMIs were delivered as regular interrupts. > But 12.2 has the newer code that delivers NMIs as NMIs. > > After some investigation it became evident that NMIs are not delivered at all. > I modified send_nmi() in sys/x86/xen/xen_apic.c to capture and report errors > from HYPERVISOR_vcpu_op(VCPUOP_send_nmi) calls. > That revealed that the call returns -38 which appears to mean ENOSYS. > > I am not sure what that could mean. > Perhaps NMI is somehow disabled in the Xen configuration (for that specific > instance type)? It's not something that you can easily disable at build time, but it's possible AWS has done some modifications to the hypervisor in order to disable it. Do you know which instance type and hypervisor version it's running on? Thanks, Roger.