From nobody Wed Dec 01 18:59:08 2021 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id AAB0718B6921; Wed, 1 Dec 2021 18:59:10 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4J47fj0DhBz3kjr; Wed, 1 Dec 2021 18:59:08 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4E23172B2; Wed, 1 Dec 2021 18:59:08 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 1B1Ix8t8060438; Wed, 1 Dec 2021 18:59:08 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 1B1Ix8HM060437; Wed, 1 Dec 2021 18:59:08 GMT (envelope-from git) Date: Wed, 1 Dec 2021 18:59:08 GMT Message-Id: <202112011859.1B1Ix8HM060437@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Alexander Motin Subject: git: 9d842d84f49a - stable/12 - mpr/mps(4): Make device mapping some more robust. List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-all@freebsd.org X-BeenThere: dev-commits-src-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: mav X-Git-Repository: src X-Git-Refname: refs/heads/stable/12 X-Git-Reftype: branch X-Git-Commit: 9d842d84f49af6d4aacaea694dcaea4173522684 Auto-Submitted: auto-generated ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1638385149; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=g5S9RYPEiSLPOc4BDZjPI3ujsB9jy6SCruz2bDUCuUM=; b=OhPL7krut48zsPUaEAjpYcOso460SUczl5NY1ZCbXeB2PeDSqu8niTG1QIp9U0qu57j7vs UUfdkKY0aZW/ehWB0kt8Al8VgxIY8kVx+lG+0hyFauulTpqsZ6pJzWCW37//ZPYa6QKDw/ FZF8t+IUYgFK/ZW61AHOfLanXGHmEfCzAjEB9HlsKqB2cfLb8efUHYB4/HgylwKebPTzbS faecr4TFO3QyGYXyACS63+e9c+rPJmWYaALR/35cpueFW6ZSgI0jOj+8ngbLHeBF3HF1iP AF1zdxDn258GVPpfbmkbvN6JbyzP/UOOrwqmY4CrJehB6vomSt45RpOth1hnVA== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1638385149; a=rsa-sha256; cv=none; b=oLmLKWVA8prdSVb53NvR+BaN340rinhvFkxS/CTQmm0ck9yJ60DCiBP9LUeXiMQmSw9wXn xj/nR8zqgIaRe8N0nztVfOl+geVzf83Z+wh9aS1DtA9bWMD9tf+ZeqDAYj74BUKvpzNamh XyGeWPk+bjTDjcmzFs0W3IxZdRNeAVBOmhx9htAzZ2FHmyz+uG8HYHyAg9RjuhJWpP3xto 6VV4Pa6mQeyb6IZpQf+f5FNMrkbx4+NBLzGD7DfmykKNBabBvuhlrEU5XkjBCEVlNYk6Kp TAsN4xA5AR266/YJJVhHvPPh6mbDq0E259+m139QBdnUfioN9gE7pvznK1cdAg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N The branch stable/12 has been updated by mav: URL: https://cgit.FreeBSD.org/src/commit/?id=9d842d84f49af6d4aacaea694dcaea4173522684 commit 9d842d84f49af6d4aacaea694dcaea4173522684 Author: Alexander Motin AuthorDate: 2021-04-24 03:18:01 +0000 Commit: Alexander Motin CommitDate: 2021-12-01 18:51:24 +0000 mpr/mps(4): Make device mapping some more robust. Allow new enclosure to replace previously existing one if there is no completely unused table entry, same as it is done for devices. If we can not process DPM due to corruption -- wipe it and restart from scratch. Otherwise I don't see a way to recover persistence if something go wrong and there is no BIOS to recover it for us. Together this solves a problem that appeared when 9300-8i firmware update to 16.00.10.00 somehow switched its mapping mode from Device Persistence to Enclosure/Slot without wiping the DPM table. It made HBA completely unusable, since overflowed and conflicting mapping table was unable to map any of enclosures and so devices. Also while there make some enclosure mapping errors more informative. MFC after: 1 month Sponsored by: iXsystems, Inc. (cherry picked from commit b99419aee49e2cc53747730be4d0ec4f9b330eb2) --- sys/dev/mpr/mpr_mapping.c | 71 ++++++++++++++++++++++++++++++----------------- sys/dev/mps/mps_mapping.c | 39 +++++++++++++++++--------- 2 files changed, 71 insertions(+), 39 deletions(-) diff --git a/sys/dev/mpr/mpr_mapping.c b/sys/dev/mpr/mpr_mapping.c index 9af67b6250f3..eed59ff38de1 100644 --- a/sys/dev/mpr/mpr_mapping.c +++ b/sys/dev/mpr/mpr_mapping.c @@ -1214,9 +1214,10 @@ _mapping_get_dev_info(struct mpr_softc *sc, phy_change->is_processed = 1; mpr_dprint(sc, MPR_ERROR | MPR_MAPPING, "%s: " "failed to add the device with handle " - "0x%04x because the enclosure is not in " - "the mapping table\n", __func__, - phy_change->dev_handle); + "0x%04x because enclosure handle 0x%04x " + "is not in the mapping table\n", __func__, + phy_change->dev_handle, + topo_change->enc_handle); continue; } if (!((phy_change->device_info & @@ -1369,9 +1370,10 @@ _mapping_get_pcie_dev_info(struct mpr_softc *sc, port_change->is_processed = 1; mpr_dprint(sc, MPR_ERROR | MPR_MAPPING, "%s: " "failed to add the device with handle " - "0x%04x because the enclosure is not in " - "the mapping table\n", __func__, - port_change->dev_handle); + "0x%04x because enclosure handle 0x%04x " + "is not in the mapping table\n", __func__, + port_change->dev_handle, + topo_change->enc_handle); continue; } if (!(port_change->device_info & @@ -1610,9 +1612,10 @@ _mapping_add_new_device(struct mpr_softc *sc, phy_change->is_processed = 1; mpr_dprint(sc, MPR_ERROR | MPR_MAPPING, "%s: " "failed to add the device with handle " - "0x%04x because the enclosure is not in " - "the mapping table\n", __func__, - phy_change->dev_handle); + "0x%04x because enclosure handle 0x%04x " + "is not in the mapping table\n", __func__, + phy_change->dev_handle, + topo_change->enc_handle); continue; } @@ -1867,9 +1870,10 @@ _mapping_add_new_pcie_device(struct mpr_softc *sc, port_change->is_processed = 1; mpr_dprint(sc, MPR_ERROR | MPR_MAPPING, "%s: " "failed to add the device with handle " - "0x%04x because the enclosure is not in " - "the mapping table\n", __func__, - port_change->dev_handle); + "0x%04x because enclosure handle 0x%04x " + "is not in the mapping table\n", __func__, + port_change->dev_handle, + topo_change->enc_handle); continue; } @@ -2208,7 +2212,7 @@ mpr_mapping_free_memory(struct mpr_softc *sc) free(sc->dpm_pg0, M_MPR); } -static bool +static void _mapping_process_dpm_pg0(struct mpr_softc *sc) { u8 missing_cnt, enc_idx; @@ -2335,8 +2339,8 @@ _mapping_process_dpm_pg0(struct mpr_softc *sc) MPR_DPM_BAD_IDX) { mpr_dprint(sc, MPR_ERROR | MPR_MAPPING, "%s: Conflict in mapping table for " - " enclosure %d\n", __func__, - enc_idx); + "enclosure %d device %d\n", + __func__, enc_idx, map_idx); goto fail; } physical_id = @@ -2376,20 +2380,26 @@ _mapping_process_dpm_pg0(struct mpr_softc *sc) mt_entry->device_info = MPR_DEV_RESERVED; } } /*close the loop for DPM table */ - return (true); + return; fail: + mpr_dprint(sc, MPR_ERROR | MPR_MAPPING, "%s: " + "Wiping DPM to start from scratch\n", __func__); + dpm_entry = (Mpi2DriverMap0Entry_t *) ((uint8_t *) sc->dpm_pg0 + + sizeof(MPI2_CONFIG_EXTENDED_PAGE_HEADER)); + bzero(dpm_entry, sizeof(Mpi2DriverMap0Entry_t) * sc->max_dpm_entries); for (entry_num = 0; entry_num < sc->max_dpm_entries; entry_num++) { + sc->dpm_flush_entry[entry_num] = 1; sc->dpm_entry_used[entry_num] = 0; /* * for IR firmware, it may be necessary to wipe out * sc->mapping_table volumes tooi */ } + _mapping_flush_dpm_pages(sc); for (enc_idx = 0; enc_idx < sc->num_enc_table_entries; enc_idx++) _mapping_clear_enc_entry(sc->enclosure_table + enc_idx); sc->num_enc_table_entries = 0; - return (false); } /* @@ -2629,10 +2639,8 @@ retry_read_dpm: } } - if (sc->is_dpm_enable) { - if (!_mapping_process_dpm_pg0(sc)) - sc->is_dpm_enable = 0; - } + if (sc->is_dpm_enable) + _mapping_process_dpm_pg0(sc); if (! sc->is_dpm_enable) { mpr_dprint(sc, MPR_MAPPING, "%s: DPM processing is disabled. " "Device mappings will not persist across reboots or " @@ -2828,14 +2836,24 @@ mpr_mapping_enclosure_dev_status_change_event(struct mpr_softc *sc, * and the enclosure has enough space in the Mapping * Table to map its devices. */ - enc_idx = sc->num_enc_table_entries; - if (enc_idx >= sc->max_enclosures) { + if (sc->num_enc_table_entries < sc->max_enclosures) { + enc_idx = sc->num_enc_table_entries; + sc->num_enc_table_entries++; + } else { + enc_idx = _mapping_get_high_missing_et_idx(sc); + if (enc_idx != MPR_ENCTABLE_BAD_IDX) { + et_entry = &sc->enclosure_table[enc_idx]; + _mapping_add_to_removal_table(sc, + et_entry->dpm_entry_num); + _mapping_clear_enc_entry(et_entry); + } + } + if (enc_idx == MPR_ENCTABLE_BAD_IDX) { mpr_dprint(sc, MPR_ERROR | MPR_MAPPING, "%s: " "Enclosure cannot be added to mapping " "table because it's full.\n", __func__); goto out; } - sc->num_enc_table_entries++; et_entry = &sc->enclosure_table[enc_idx]; et_entry->enc_handle = le16toh(event_data-> EnclosureHandle); @@ -2862,8 +2880,9 @@ mpr_mapping_enclosure_dev_status_change_event(struct mpr_softc *sc, le16toh(event_data->EnclosureHandle)); if (enc_idx == MPR_ENCTABLE_BAD_IDX) { mpr_dprint(sc, MPR_ERROR | MPR_MAPPING, "%s: Cannot " - "unmap enclosure %d because it has already been " - "deleted.\n", __func__, enc_idx); + "unmap enclosure with handle 0x%04x because it " + "has already been deleted.\n", __func__, + le16toh(event_data->EnclosureHandle)); goto out; } et_entry = &sc->enclosure_table[enc_idx]; diff --git a/sys/dev/mps/mps_mapping.c b/sys/dev/mps/mps_mapping.c index 9330288acd06..d2f761568dac 100644 --- a/sys/dev/mps/mps_mapping.c +++ b/sys/dev/mps/mps_mapping.c @@ -1176,9 +1176,10 @@ _mapping_get_dev_info(struct mps_softc *sc, phy_change->is_processed = 1; mps_dprint(sc, MPS_ERROR | MPS_MAPPING, "%s: " "failed to add the device with handle " - "0x%04x because the enclosure is not in " - "the mapping table\n", __func__, - phy_change->dev_handle); + "0x%04x because enclosure handle 0x%04x " + "is not in the mapping table\n", __func__, + phy_change->dev_handle, + topo_change->enc_handle); continue; } if (!((phy_change->device_info & @@ -1421,9 +1422,10 @@ _mapping_add_new_device(struct mps_softc *sc, phy_change->is_processed = 1; mps_dprint(sc, MPS_ERROR | MPS_MAPPING, "%s: " "failed to add the device with handle " - "0x%04x because the enclosure is not in " - "the mapping table\n", __func__, - phy_change->dev_handle); + "0x%04x because enclosure handle 0x%04x " + "is not in the mapping table\n", __func__, + phy_change->dev_handle, + topo_change->enc_handle); continue; } @@ -1887,8 +1889,8 @@ _mapping_process_dpm_pg0(struct mps_softc *sc) MPS_DPM_BAD_IDX) { mps_dprint(sc, MPS_ERROR | MPS_MAPPING, "%s: Conflict in mapping table for " - " enclosure %d\n", __func__, - enc_idx); + "enclosure %d device %d\n", + __func__, enc_idx, map_idx); break; } physical_id = @@ -2364,14 +2366,24 @@ mps_mapping_enclosure_dev_status_change_event(struct mps_softc *sc, * and the enclosure has enough space in the Mapping * Table to map its devices. */ - enc_idx = sc->num_enc_table_entries; - if (enc_idx >= sc->max_enclosures) { + if (sc->num_enc_table_entries < sc->max_enclosures) { + enc_idx = sc->num_enc_table_entries; + sc->num_enc_table_entries++; + } else { + enc_idx = _mapping_get_high_missing_et_idx(sc); + if (enc_idx != MPS_ENCTABLE_BAD_IDX) { + et_entry = &sc->enclosure_table[enc_idx]; + _mapping_add_to_removal_table(sc, + et_entry->dpm_entry_num); + _mapping_clear_enc_entry(et_entry); + } + } + if (enc_idx == MPS_ENCTABLE_BAD_IDX) { mps_dprint(sc, MPS_ERROR | MPS_MAPPING, "%s: " "Enclosure cannot be added to mapping " "table because it's full.\n", __func__); goto out; } - sc->num_enc_table_entries++; et_entry = &sc->enclosure_table[enc_idx]; et_entry->enc_handle = le16toh(event_data-> EnclosureHandle); @@ -2398,8 +2410,9 @@ mps_mapping_enclosure_dev_status_change_event(struct mps_softc *sc, le16toh(event_data->EnclosureHandle)); if (enc_idx == MPS_ENCTABLE_BAD_IDX) { mps_dprint(sc, MPS_ERROR | MPS_MAPPING, "%s: Cannot " - "unmap enclosure %d because it has already been " - "deleted.\n", __func__, enc_idx); + "unmap enclosure with handle 0x%04x because it " + "has already been deleted.\n", __func__, + le16toh(event_data->EnclosureHandle)); goto out; } et_entry = &sc->enclosure_table[enc_idx];