FreeBSD 9 and ARP multicast source address error messages

[Alexander Wittig]

Gleb,

Am 10.11.2011 um 01:51 schrieb Gleb Smirnoff:

> On Tue, Nov 08, 2011 at 05:14:45PM -0500, Alexander Wittig wrote:
> A> I upgraded one of my machines from FreeBSD 8 to 9.0-RC1 (FreeBSD bt.pa.msu.edu 9.0-RC1 FreeBSD 9.0-RC1 #3: Fri Oct 28 16:45:28 EDT 2011     root at bt.pa.msu.edu:/usr/obj/usr/src/sys/ALEX  i386), and ever since that upgrade the kernel keeps flooding my log files with messages like these:
> A> Nov  7 16:40:01 bt kernel: in_arp: source hardware address is multicast.in_arp: source hardware address is multicast.
> A> Nov  7 16:42:02 bt kernel: in_arp: source hardware address is multicast.in_arp: source hardware address is multicast.
> A> 
> A> A Google search for these didn't reveal any useful results as to why this happens or how to fix it. So I did a tcpdump and matched the time stamps with packets, and I found the ones causing problems (the only ones with a multicast bit set) to be like this:
> A> 16:40:01.099823 02:02:23:09:44:3c > 03:bf:23:09:44:87, ethertype ARP (0x0806), length 60: Ethernet (len 6), IPv4 (len 4), Reply 35.9.68.228 is-at 03:bf:23:09:44:e4, length 46
> A>         0x0000:  03bf 2309 4487 0202 2309 443c 0806 0001
> A>         0x0010:  0800 0604 0002 03bf 2309 44e4 2309 44e4
> A>         0x0020:  02bf 2309 443c 2309 4487 0000 0000 0000
> A>         0x0030:  0000 0000 0000 0000 0000 0000
> A> 
> A> It appears the broadcast MAC 03:bf:23:09:44:87 is part of Microsoft's network load balancing mechanism, with the 03:bf indicating that much and the 23:09:44:87 containing the IP address of the load balance cluster (35.9.68.228). These types of MACs seem to be commonly used in their load balancing implementation.
> A> 
> A> To prevent these messages from producing thousands of lines of logs each day, I added the following two IPFW rules and enabled ethernet package filtering (sysctl net.link.ether.ipfw=1):
> A> deny ip from any to any MAC 03:bf:00:00:00:00/16 any layer2
> A> allow ip from any to any layer2
> A> 
> A> This effectively blocks those packages and the resulting error messages. But I'm wondering if the newly added(?) ARP code in FBSD 9 is a bit too fussy about these, or if MS is abusing the ARP protocol here. Either way, this was never a problem with FBSD 7 or 8.
> 
> Can you try attached patch. It reduces severity level of all ARP
> messages, that can be triggered by packet on network, with expection to
> "using my IP address".
> 
> With default syslog.conf, now ARP errors won't get to console.
> 
> Also, the multicast message lacked "\n" newline character, that's why,
> I suppose, syslogd failed to coalesce a number of messages into a single
> entry "last message repeated X times".

Thank you very much for the patch, and for making this particular message a bit more helpful by including the MAC address.
I tried it and indeed it stops the messages from going to the console. But the default syslog.conf still logs each one in /var/log/messages and they also show up in dmsg output. These happen quite frequently, so even on a not so busy network they will drown out almost everything else going on in dmsg or /var/log/messages. Unfortunately, having two (or more) different machines use these will prevent syslogd from combining messages into "last message repeated X times":

/var/log/messages:
[...]
Nov 10 12:23:44 bt kernel: in_arp: 03:bf:23:09:44:e4 is multicast
Nov 10 12:23:44 bt kernel: in_arp: 03:bf:23:09:44:87 is multicast
Nov 10 12:25:45 bt kernel: in_arp: 03:bf:23:09:44:e4 is multicast
Nov 10 12:25:45 bt kernel: in_arp: 03:bf:23:09:44:87 is multicast
[...]

I'm not an expert on networking, but is this condition of ignoring such an ARP packet really a noteworthy event? I.e. is this something that should not occur in "normal" operation according to the ARP specifications? If this is mostly for kernel developers, maybe it should only be enabled in debug kernel builds?

Alex