divert and deadlock issues

[Bruce M. Simpson]

Christian S.J. Peron wrote:
> ...
> One idea was to duplicate the socket options mbuf and pass in a NULL pointer
> for the multi-cast options.  Keep in mind that these are multicast options
> associated with a divert socket.
>
> So I guess the questions:
>
> (1) Are there any users that are specifying multicast options on divert sockets?
> (2) Are there any users that are specifying socket options in general for
>     divert sockets?
>   

The LOR is obviously being triggered by ip_output()'s acquisition of 
in_multi_mtx, due to a datagram being sent to a multicast destination 
and a subsequent lookup being required.

I can't think of a reason why a user would wish to supply any multicast 
socket options to a divert socket, other than the 'small' ones, i.e. 
IP_MULTICAST_TTL/IF/LOOP/VIF.

See the comments about idempotence inside in_mcast.c on the HEAD branch, 
about why you can't just wish them away. It seems reasonable that this 
subset of the multicast options are supported for divert sockets given 
the likely use cases, even if IPPROTO_DIVERT supports IP_HDRINCL, 
because IP_MULTICAST_TTL does not do what you think it does (see 
in_mcast.c comments again).

Joining groups on a divert socket SHOULD NOT be supported (it does not 
make sense semantically) and we should deliberately return EINVAL for 
multicast options other than the above subset.

Dropping the inpcb lock over ip_output() looks like the easy option. 
Alternatively, we could just not support multicast options on divert 
sockets given that it is a rare use case as per above.

BMS