IP_MAX_MEMBERSHIPS story.

[Bruce M Simpson]

A user recently reported a problem with running into IP_MAX_MEMBERSHIPS
on a system running FreeBSD with IPv4 forwarding enabled, and running
the OSPF routing protocol.

I have been investigating how to address this problem.

Background:

 A raw socket was exceeding the permitted number of group memberships.

 Because the socket layer, and not the link layer, was being used for
 the transmission and reception of multicast IPv4 traffic, the use of
 multicast-promiscuous mode is not applicable (except where the underlying
 NIC driver would enable it to deal with running out of multicast hash
 filter entries).

 IPv6 is not affected - read on.

Analysis:

 The reason for this is quite simple.

 struct ip_moptions contains a field after imo_memberships (the array
 which is statically sized by IP_MAX_MEMBERSHIPS).
 This limit is per-socket, *not* per-interface.
 This is consistent with the description of the code given in TCP/IP
 Illustrated Volume 2 (Wright/Stevens), section 12.7.

 However, because we now have a field which is present after this array,
 referenced by the IGMP code and the IPv4 multicast routing code, a
 simple expansion (run-time or otherwise) of the array is not adequate
 to solve the problem.

 The way KAME avoided this in IPv6 was to hold IPv6 group memberships
 in a doubly linked list, which is probably acceptable given that these
 structures are traversed on multicast input/output and socket option
 manipulation.

 Both NetBSD and OpenBSD are also affected by this issue, and potentially
 also Darwin and MacOS X.

 The way Linux has avoided this is by having the network code structured
 completely differently.

Resolution:

 1. Rearrange struct ip_moptions in netinet/ip_var.h such that the
    IP_MAX_MEMBERSHIPS value, and thus the size of the imo_membership
    array, may be adjusted via a boot-time tunable.
    A relatively easy change but one that still breaks the ABI.
Or;
 2. Change semantics of imo_membership to match those of netinet6, by
    using a linked list.
    A somewhat more involved change but one that still breaks the ABI.

The loadable kernel modules directly affected by the ABI breakage
seem limited only to ip_mroute_mod.ko.

Comments? Suggestions?

BMS