arplookup WWW.XXX.YYY.ZZZ failed: host is not on local network

[David Fuchs]

Ok, riddle me this:

/kernel: arplookup WWW.XXX.YYY.10 failed: host is not on local network

I've been seeing these messages in the messages log, so I've done some 
additional tests and am now hunting for answers.

The curious part is, the host technically *is* on the local network. 
It's a /24 subnet; the address of the local machine is WWW.XXX.YYY.25 
and the other host is WWW.XXX.YYY.10.  As I mentioned, this is a /24, 
and so I've confirmed that both hosts are configured with the subnet 
mask of 0xffffff00 (255.255.255.0).

I can repeat the error message as often as I want by generating arp 
who-has requests (I've been using `arping' to accomplish these tests):

tcpdump output when running arping:

arp who-has WWW.XXX.YYY.10 tell WWW.XXX.YYY.25
arp reply WWW.XXX.YYY.10 is-at 0:e0:81:2:15:59

At the same time, my messages log fills up with arplookup errors.

/kernel: arplookup WWW.XXX.YYY.10 failed: host is not on local network

Now, I must add a necessary development to this, which is almost 
definitely the cause of the problem.  These two hosts each have two 
interfaces on them.  The primary interface on each is connected to 
WWW.XXX.YYY.0/24 as shown above.  The secondary interface on each is 
assigned RFC1918 addresses on the 172.16.1.0/24 subnet, and are directly 
connected with a crossover cable.   The last octet used in the IP 
addresses for the secondary interfaces mirror the last octet used for 
the IP on the primary interfaces.  Static routes have been added to 
force all communication *between* these two hosts to use the secondary 
interfaces:

WWW.XXX.YYY.25's static route:
	route add WWW.XXX.YYY.10 172.16.1.10

WWW.XXX.YYY.10's static route:
	route add WWW.XXX.YYY.25 172.16.1.25

The static routes work as they should - traffic destined for either 
host's IP on the WWW.XXX.YYY.0/24 subnet goes through the secondary 
interfaces instead.  However, both machines are receiving the arplookup 
errors because of this, and I'd like to know if there is a solution 
besides disabling the sysctl variable that causes these errors to be logged.

Thanks!

-David Fuchs