Bridge Interfaces and ARPs

[Jason Van Patten]

Hey gang -

I posted this to the FreeBSD user forums but figured I'd send a message 
off to the list to see if anyone has any input, guidance, or ideas. 
Emailing diagrams around isn't good form (IMHO) but having a diagram 
handy will help with the discussion.  So please glance at:

http://pics.lateapex.net/vz.png

Background: I have a business class Verizon FIOS connection for Internet 
at home.  Along with that connection, I have 13 (not 14!) static IPs 
from VZ.  They almost fall within a proper CIDR block, but not quite: 
1.2.3.210 - 1.2.3.222.  I don't own .209, so I can't claim 1.2.3.208/28 
as my IP block (dammit!)  The subnet for the static IPs is a /24, and 
the default route is *Verizon's* router: 1.2.3.1.

There are a number of different choices for this network layout: DMZ, 
bridging, or binat.  I chose bridging so that I don't have the 
complexity of binatting, and yet have some protection for the servers 
via my router.  So, per the drawing, the FreeBSD router's em0 is 
connected to the Verizon equipment, while re0 and re1 are both connected 
to a managed Cisco switch, on different VLANs.

VLAN 10 for re0: Public IPs (public services, etc)
VLAN 20 for re1: Private IPs (NAS, wireless AP, etc)

Via the router, VLAN 10 and Verizon's network are bridged together.  The 
bridge interface on the router has IP: 1.2.3.222/24 with a default route 
set to 1.2.3.1.  All servers on VLAN 10 have IPs within the allocated 
range (.210 - .220) and the same default route.

Now: the problem.  I used the LAGG'd server as an example in the 
diagram, but the same thing is happening with other servers: the router 
is learning ARP entries for the IPs I own *from* Verizon's router.  As 
soon as the router caches that bad entry, it no longer routes traffic to 
those public IPs *from* VLAN 20 (private side).  So, in other words, a 
laptop on the wireless network won't be able to get to 1.2.3.215.

My work-around for now has been a series of static ARP entries on the 
router for each of my public servers.  That seems to work fine, but I 
wonder if there's something I might be doing wrong?

If I didn't include enough info, fire away.  Thanks!

-- 
Jason Van Patten