rev. 1.94 of netinet/in.c broke CARP

[Bruce M. Simpson]

Gleb,

Good catch, thanks for tracking this down.

Gleb Smirnoff wrote:
>   I've just discovered, that revision 1.94 of in.c has broke CARP. This
> change adds a code to in_ifdetach() that goes through the global list
> of all multicast instances and deletes all the instances, that are
> belonging to a particular interface. This is intended to avoid leaking
> multicast instances.
>   
The irony of this of course is that I was working on two separate fixes; 
one to prevent the MROUTING code panicking when an interface was 
suddenly removed, and the other to prevent the netinet ifp detach path 
from panicking in the same circumstances.

These are resource leaks which have been in BSD for years and years. I 
neglected to test them *together* however; carp(4) was being used in the 
test cases for both bugs.

>   Before this change, most of the subsystems, that allocated multicast
> membership instances had freed is theirselves. I don't know about others,
> but at least CARP is broken now. It attempts to free a memory, that
> already has been freed.
>   
I would suggest that the correct fix, for now, would be for carp(4) to 
now *not* perform its own cleanup for the IPv4 groups it joins on member 
interfaces.

The symptom here is that carp(4) needs to join a multicast group on its 
member interface. When the interface goes away, the group membership is 
now destroyed, at the netinet global level, by the netinet detach path 
first.

However, carp(4) is keeping its own imo_membership vector of the 
addresses it joined on its member interfaces (rather than using the one 
which netinet assigns to it in its attach path), and later tries to free 
these memberships.

netinet6 does not have the same problem because in6 memberships are 
reference counted.

The root problem is that we should be using consistent semantics for 
both the IPv4 and IPv6 paths, and the kernel APIs where soft-ifnets 
(such as carp(4)) and routing code (such as MROUTING) need to manipulate 
multicast group memberships.

Regards,
BMS