git: 5b59c5186902 - stable/12 - nvme: use config_intrhook_drain to avoid removable card races

[Warner Losh]

The branch stable/12 has been updated by imp:

URL: https://cgit.FreeBSD.org/src/commit/?id=5b59c5186902f8ac402b89796567fa630abb10c2

commit 5b59c5186902f8ac402b89796567fa630abb10c2
Author:     Warner Losh <imp at FreeBSD.org>
AuthorDate: 2021-03-11 15:42:44 +0000
Commit:     Warner Losh <imp at FreeBSD.org>
CommitDate: 2021-07-31 00:02:53 +0000

    nvme: use config_intrhook_drain to avoid removable card races
    
    nvme drives are configured early in boot. However, a number of the configuration
    steps takes which take a while, so we defer those to a config intrhook that runs
    before the root filesystem is mounted. At the same time, the PCI hot plug wakes
    up and tests the status of the card. It may decide that the card has gone away
    and deletes the child. As part of that process nvme_detach is called. If this
    call happens after the config_intrhook starts to run, but before it is finished,
    there's a race where we can tear down the device's soft state while the
    config_intrhook is still using it. Use the new config_intrhook_drain to
    disestablish the hook. Either it will be removed w/o running, or the routine
    will wait for it to finish. This closes the race and allows safe hotplug at any
    time, even very early in boot.
    
    Sponsored by:           Netflix, Inc
    Reviewed by:            jhb, mav
    Differential Revision:  https://reviews.freebsd.org/D29006
    
    (cherry picked from commit 8423f5d4c127f18e7500bc455bc7b6b1691385ef)
---
 sys/dev/nvme/nvme.c       | 5 +----
 sys/dev/nvme/nvme_ctrlr.c | 2 --
 2 files changed, 1 insertion(+), 6 deletions(-)

diff --git a/sys/dev/nvme/nvme.c b/sys/dev/nvme/nvme.c
index 8e5bc11b35d4..17ee7b19177d 100644
--- a/sys/dev/nvme/nvme.c
+++ b/sys/dev/nvme/nvme.c
@@ -144,10 +144,7 @@ nvme_detach(device_t dev)
 {
 	struct nvme_controller	*ctrlr = DEVICE2SOFTC(dev);
 
-	if (ctrlr->config_hook.ich_arg != NULL) {
-		config_intrhook_disestablish(&ctrlr->config_hook);
-		ctrlr->config_hook.ich_arg = NULL;
-	}
+	config_intrhook_drain(&ctrlr->config_hook);
 
 	nvme_ctrlr_destruct(ctrlr, dev);
 	return (0);
diff --git a/sys/dev/nvme/nvme_ctrlr.c b/sys/dev/nvme/nvme_ctrlr.c
index d4d472878b82..007e0758023a 100644
--- a/sys/dev/nvme/nvme_ctrlr.c
+++ b/sys/dev/nvme/nvme_ctrlr.c
@@ -1135,7 +1135,6 @@ nvme_ctrlr_start_config_hook(void *arg)
 fail:
 		nvme_ctrlr_fail(ctrlr);
 		config_intrhook_disestablish(&ctrlr->config_hook);
-		ctrlr->config_hook.ich_arg = NULL;
 		return;
 	}
 
@@ -1154,7 +1153,6 @@ fail:
 
 	nvme_sysctl_initialize_ctrlr(ctrlr);
 	config_intrhook_disestablish(&ctrlr->config_hook);
-	ctrlr->config_hook.ich_arg = NULL;
 
 	ctrlr->is_initialized = 1;
 	nvme_notify_new_controller(ctrlr);