devfs panic w/INVARIANTS

[Kostik Belousov]

On Thu, Feb 04, 2010 at 03:40:28PM -0500, Andrew Gallatin wrote:
> I've got a commercial driver that uses device cloning.
> At unload time, the driver calls clone_cleanup(). When I unload
> the driver when the kernel is built with INVARIANTS, I'll see a
> panic in devfs_populate_loop().  This happens in 6-stable,
> as well as 8-stable.
> 
> From what I can see the clone has been freed, but it
> remains on the devfs cdevp_list.   Then the next time
> devfs_populate_loop() is called, it trips over the bad
> entry (cdp->cdp_dirents points to 0xdeadc0dedeadc0de)
> See appended kgdb session.
> 
> If I trace the code path, it looks like clone_cleanup()
> calls destroy_devl().  And destroy_devl() will eventually
> call devfs_free() if the si_refcnt is zero.  But I don't
> see anything which will get the cdev removed from
> the cdevp_list prior to it being freed.
> 
> The only code I see which will get the cdev removed from
> the cdevp_list() seems to be the "GC any lingering devices"
> block in devfs_populate_loop
> 
> What am I missing?

You did not mentioned it, but my guess is that you create clones from
the dev_clone event handler. Please note that devfs_lookup() that fires
dev_clone event, consumes a device reference. Thus clone handlers shall
do dev_ref().

Due to races with cleanup, you should use MAKEDEV_REF flag for
make_dev_credv(9) KPI instead of doing make_dev()/dev_ref() pair.

That said, do you really need clones at all ?
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 196 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20100205/0a6308c9/attachment.pgp