Panic on reloading a driver with same DEVICE_PROBE() return value

[Sreekanth Reddy]

Hi,

We are trying to rename our out-of-box driver from "mpslsi3" to "mpr"
(which is the inbox driver name).  From in-box to out-of-box driver
diff is vary small.

Here are the steps to reproduce this issue,

1. Replace the in-box mpr.ko file from out-of-box's mpr.ko file in
/boot/kernel/ path
2. Added below line in the file /boot/loader.conf
     mpr_load="YES"
3. Then rebooted the server and verified that out-of-box driver is
loaded during the boot time.
4. Then unloaded the mpr driver using "kldunload mpr" command.
5. Now when I reload the driver using "kldload mpr" then we observed below panic

--------------------------------------------------------------------------------------------------------------------------------------------
Unread portion of the kernel message buffer:
mpr0: <Avago Technologies (LSI) SAS3008> port 0x8000-0x80ff mem
0xdfe40000-0xdfe4ffff,0xdfe00000-0xdfe3ffff irq 32 at device 0.0 on
pci1


Fatal trap 12: page fault while in kernel mode
cpuid = 2; apic id = 04
fault virtual address   = 0x8
fault code              = supervisor read data, page not present
instruction pointer     = 0x20:0xffffffff8092adf9
stack pointer           = 0x28:0xfffffe085d0cb2b0
frame pointer           = 0x28:0xfffffe085d0cb300
code segment            = base 0x0, limit 0xfffff, type 0x1b
                        = DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags        = interrupt enabled, resume, IOPL = 0
current process         = 770 (kldload)
trap number             = 12
panic: page fault
cpuid = 2
KDB: stack backtrace:
#0 0xffffffff80984e30 at kdb_backtrace+0x60
#1 0xffffffff809489e6 at vpanic+0x126
#2 0xffffffff809488b3 at panic+0x43
#3 0xffffffff80d4aadb at trap_fatal+0x36b
#4 0xffffffff80d4addd at trap_pfault+0x2ed
#5 0xffffffff80d4a47a at trap+0x47a
#6 0xffffffff80d307f2 at calltrap+0x8
#7 0xffffffff819a0930 at _end+0x2a7d00
#8 0xffffffff8199e5ba at _end+0x2a598a
#9 0xffffffff8097bbfd at device_attach+0x43d
#10 0xffffffff8069385a at pci_driver_added+0xea
#11 0xffffffff80979d3a at devclass_driver_added+0x7a
#12 0xffffffff80979c9c at devclass_add_driver+0x11c
#13 0xffffffff8092d50b at module_register_init+0xfb
#14 0xffffffff80922bf7 at linker_load_module+0xc07
#15 0xffffffff80924043 at kern_kldload+0xc3
#16 0xffffffff8092411b at sys_kldload+0x5b
#17 0xffffffff80d4b3f7 at amd64_syscall+0x357
------------------------------------------------------------------------------------------------------------------------------------------

Note:
* Same panic occurs, even if we unload and load the in-box mpr driver.
* Now both in-box and in out-of-box drivers return with
"BUS_PROBE_DEFAULT" from device_probe() callback function. we won't
observe any panic if we reload the the driver with higher
device_probe() return value when compared with the drive's
device_probe() return value which has loaded during the OS boot time.
For example:
  1. First booted the OS with the out-of-box driver which returns with
"BUS_PROBE_DEFAULT" device_probe() return value.
 2. Then I have unloaded this out-of-box driver using kldunload command,
 3. Then in driver code, I have modified the device_probe()'s return
value to "BUS_PROBE_VENDOR" and then compiled and loaded this
experimented driver successfully with out any panic.
 4. Again I am observing same issue if I reboot the OS with this
experimented driver loaded a boot time and if reload the same
experimented driver.

I found that panic occurs while first time allocating memory with
malloc() API, In mpr driver, it occurs while executing below line
while reloading the same driver,

sc->facts = malloc(sizeof(MPI2_IOC_FACTS_REPLY), M_MPR, M_ZERO|M_NOWAIT);


Here are my quires:
* Why driver reload will successful only if the device_probe()'s
return value is higher then the driver which has loaded during OS boot
time. Whether kernel will save device_probe()'s return value of the
driver which is loaded during the OS boot time.
* How can I reload the modified driver with return same device_probe()
value as the one which has loaded during the OS boot time. I observed
that whenever I try to load this modified driver using "kldload
./mpr.ko" command then I see that driver in the /boot/kernel/ path has
executed.

Thanks,
Sreekanth