IPFW and VLANs

[Brett Glass]

Everyone:

I'm writing some new rulesets for IPFW on a machine that has only 
one built-in Ethernet interface. It connects to a few different 
Ethernets via a VLAN switch. (The physical interface leads to a 
"trunk;" that is to say, all packets passing om and out of the 
parent interface ought to be tagged with a VLAN number associated 
with one or more of the ports on the external switch. There 
shouldn't be any untagged packets on the interface.)

One of the things I wanted to do in my rules was block IPv4 
multicast packets on some -- or possibly all -- of the interfaces, 
depending on whether an interface needed to use routing protocols 
that did multicasting. I became curious: When there are VLANs 
(which are implemented as "child" interfaces in FreeBSD), is each 
Layer 2 packet that passes through the "parent" interface handed to 
IPFW twice -- once for the parent interface and once for the child 
for which the packet is tagged? (This would be inefficient, but at 
least I could minimize the inefficiency by putting in an early rule 
to pass all packets on the parent and then filter on the 
children... or just filter on the parent if I wanted to block all 
multicasting.)

Figuring that it would be fastest to do an empirical test to see 
how the packets were handled, I set the sysctl variable 
net.link.ether.ipfw to 1 and set up some rules to check the 
behavior. The rules counted all of the raw (layer2) packets on the 
parent interface (re0) and also on one of the children (re0_1); one 
of them also looked for non-Layer 2 traffic on the parent (which I 
didn't expect to find). I then let the machine, which was set up as 
a router, process a bit of traffic. What I saw, when I looked at 
the results, was downright strange:

00001  4290 1268452 count ip from any to any layer2 via re0_1
00002  3878 1251586 count ip from any to any layer2 via re0
00003     0       0 count ip from any to any not layer2 via re0

According to these counts, IPFW wasn't getting Layer 3 packets from 
the parent interface (Rule 3). That made sense, because the parent 
did not even have an IP address assigned to it. However, IPFW 
seemed to be counting more packets passing through one of the 
"child" interfaces (Rule 1) than through the parent (Rule 2), even 
though other "child" interfaces were also quite active.

I added a few more rules, with "recv" and "xmit" options, and 
checked the counts again after zeroing them and letting the router 
run for a bit:

00001 20591  8769298 count ip from any to any layer2 via re0_1
00002 18715  8725085 count ip from any to any layer2 via re0
00003     0        0 count ip from any to any not layer2 via re0
00004 18715  8725085 count ip from any to any layer2 recv re0
00005 18715  8725085 count ip from any to any layer2 xmit re0
00006 12746  1324342 count ip from any to any layer2 recv re0_1
00007 20592  8770798 count ip from any to any layer2 xmit re0_1

Maybe I am missing something (as I often do), but this seems just plain wrong.

What gives? Help in interpreting these results would be much appreciated.

--Brett Glass