ipv6 multicast refcnt panic

[Bruce M. Simpson]

I speculate that the problem you are seeing in netinet6 is due to it not 
freeing referenced in6_multi objects when the interface address changes 
or the same address is re-added, as the same bug was present in netinet. 
Previous to the introduction of refcounting, FreeBSD would just leak memory.

Further to this:

The problem Yar was seeing with vlan and pfsync, which I pointed out, 
was an older bug which has been progressively shuffled around the stack 
due to code rewrites. I have a fix for the kernel panic caused by 
pfsync's member interface being detached which is now checked into 
bms_netdev, it should probably go straight into -CURRENT.

The fix is cumulative -- pfsync's detach handler is called after netinet 
has torn down all inet state for an instance of ifnet, therefore it 
should not be trying to call in_delmulti(), however it should mark the 
ifp as no longer valid for pfsync's use.

A suggested architectural fix going forward, is to change the semantics 
of objects owned by the netinet and netinet6 protocol domains, such as 
multicast group objects, to tear down hardware state when the ifnet 
instance goes away, yet allow consumers elsewhere in the kernel to 
retain handles for such objects. This is what the lower-level net code 
now does for ifmultiaddr objects.

if_delmulti_locked() accepts an argument which specifies whether it is 
being called from if_detach(). If so, hardware state is torn down, and 
internal structures are freed, but the object *is not* freed if its 
reference count is not zero as someone still holds a pointer.

In plainer language: netinet and netinet6 should probably be doing the 
same thing as net now does, insofaras this only apples to ifmultiaddr, 
the same should be done for in_multi and in6_multi.

Of course, it would be easier to do this if per-protocol-domain state in 
ifnet were e.g. moved to the if_afdata[] array currently defined in 
ifnet for this purpose, this is guaranteed to break the ABI. The 
situation in ifnet as it stands just now strikes me as one of confusion.

Regards,
BMS