Unexpected multicast IPv4 socket behavior

[Bruce M. Simpson]

Hi,

This is ironic because I've been up against a similar problem with 
255.255.255.255 on my current project, which also requires a 'bump in 
the stack', and the same code you've posted the patch for, I found 
myself reading yesterday to answer another chap's query.

Fredrik Lindberg wrote:
> Hi
>
> I find the following socket behavior a bit unexpected. Multicast from
> an IPv4 socket (with IP_MULTICAST_IF set) with its source address bound
> to INADDR_ANY only works if there is a default route defined, otherwise
> send() returns ENETUNREACH.
>
> Default route set, src INADDR_ANY : Works
> Default route set, src bind() to interface address : Works
> No default route, src INADDR_ANY : Returns ENETUNREACH
> No default route, src bind() to interface address : Works

Totally expected behaviour. There's no way for the stack to know which 
interface to originate the traffic from in the case where there is no 
default route, and no IP layer source information elsewhere in the stack.

It could be argued that case 3 is in fact an abuse of the APIs. In IPv6, 
the use of multicast requires that you create a socket and bind to the 
interface where you wish to send and receive the channel. This is 
reasonable because both IGMP and MLD require that their group state 
traffic is bound to a specific address. Thus the glaring holes in IGMP 
due to the lack of IPv4 link-local addressing.

The newer multicast APIs in fact require you to do this, precisely to 
avoid this ambiguity. As such IP_MULTICAST_IF should be considered 
legacy -- however -- as we've seen, there's a lack of knowledge out 
there about exactly how this stuff is supposed to work.

>
> In all cases IP_MULTICAST_IF was set to the outgoing interface and
> IP_ADD_MEMBERSHIP was properly called. IGMP membership reports
> were seen on the link in all cases.

Now, if you are explicitly telling the stack which interface to use with 
IP_MULTICAST_IF, and you are seeing the regression in case 3 above, THAT 
looks like a bug.

>
> I believe the cause of this (unless this is the expected behavior?)
> is in in_pcbconnect_setup() (netinet/in_pcb.c) [1].
> The check for a multicast destination address is run after the attempt
> to get the source address by finding a directly connected interface,
> this check also returns ENETUNREACH if it fails (which it does for the
> destination 224.0.0.0/24 if no default route is set).

But but but. Sends to 224.0.0.0/*24* should never fail as it is strictly 
scoped to a link, and does not require any IPv4 route information. This 
is the lonesome kicker -- IP needs to know where to source the send 
from, however, you've told it to already with IP_MULTICAST_IF, so there 
is definitely a bug.

See the IN_LOCAL_GROUP() macro in -CURRENT's netinet/in.h for how to 
check for 224.0.0.0/24 in code.

In fact we should probably disallow multicast sends to this address when 
the socket HAS NOT been bound, except of course for the case where the 
interface is unnumbered -- but we still need a means of telling the 
stack about this case. The answer might be something called IP_SENDIF... 
Linux uses SO_BINDTODEVICE for this. It's a case of sitting down and 
doing it.

It's reasonable to assume that multicast applications should know that 
they need to walk the system's interface tree and be aware of interfaces 
and their addresses. Apps which don't do this are legacy and need to be 
updated to reflect how IP stacks actually behave now.

>
> Moving the multicast check before the directly connected check solves
> this (or any other combinations that makes sure that the
> IN_MULTICAST() check is executed).

You are quite right that the imo_multicast_ifp check needs to happen 
further up.

This is probably OK as a workaround -- but -- bigger changes need to 
happen in that code as currently source selection is mostly based on 
destination. This isn't always the case, and in multicast it certainly 
ISN'T the case as you have seen.

SO_DONTROUTE is something of a misnomer anyway. Routes still need to be 
present in the forwarding table for certain lookups, and the source 
interface selection is almost wholly based on the destination faddr in 
the inpcb, in both the cases of connect() and temporary connect during a 
sendto().

Your patch should be OK to go in. Regardless of whether there are routes 
for
the multicast channel you're using or not, IP_MULTICAST_IF is a 
sledgehammer which says 'I use THIS interface for multicast', and until 
our IPv4 stack has link scope addresses, it will be needed.

Thanks again...
BMS