Is there anything weird I should know about using ipfw on alias addresses?

[Ian Smith]

On Tue, 2 Dec 2008, Brett Davidson wrote:
 > Ian Smith wrote:
 > > On Mon, 01 Dec 2008 16:52:12 +1300 Brett Davidson <brett at net24.co.nz>
 > > wrote:
 > > 
 > >  > ifconfig shows the alias addresses correctly bound.
 > >  > Creating an ipfw rule and testing it from the command line works  >
 > > (connects out from master address, not alias)
 > >  >  >  From website on alias address, the firewall blocks the packets.
 > >  >
 > >  > The weird thing is that it tags them (in the security log) as coming  >
 > > from the master address (not the alias) out the correct interface. In a  >
 > > normal world that would mean the packet would match!!!!!
 > >  >  > What's goin' on here Willis?
 > > 
 > > Difficult to tell without seeing a) ifconfig b) netstat -rn c) at least the
 > > relevant firewall rule/s and d) log entries that illustrate your problem.
 > > Obscure sensitive information by all means, but otherwise pretend we
 > > haven't the slightest clue how your system is configured :)
 >
 > Fair enough.
 > 
 > ifconfig below:
 > 
 > bce1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
 >        options=3b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU>
 >        inet 210.5.50.5 netmask 0xffffffe0 broadcast 210.5.50.31
NB ..
 >        inet 210.5.51.32 netmask 0xffffffff broadcast 210.5.51.32
 >        inet 210.5.51.27 netmask 0xffffffff broadcast 210.5.51.27
 >        inet 210.5.51.33 netmask 0xffffffff broadcast 210.5.51.33
 >        inet 210.5.51.34 netmask 0xffffffff broadcast 210.5.51.34
 >        inet 210.5.51.42 netmask 0xffffffff broadcast 210.5.51.42
 >        inet 210.5.51.4 netmask 0xffffffff broadcast 210.5.51.4
 >        ether 00:1c:c4:c0:56:94
 >        media: Ethernet autoselect (1000baseSX <full-duplex>)
 >        status: active
 > 
 > Relevant /etc/rc.conf entries :
 > ifconfig_bce1="inet 210.5.50.5  netmask 255.255.255.224"
 > ifconfig_bce1_alias0="inet 210.5.50.5 netmask 255.255.255.224"

Your first alias here is a repeat of the 'primary' address.  ifonfig 
seems to have resolved/merged that above, but it's not an alias.

 > ifconfig_bce1_alias1="inet 210.5.51.4 netmask 255.255.255.255"
 > ifconfig_bce1_alias2="inet 210.5.51.27 netmask 255.255.255.255"
 > ifconfig_bce1_alias3="inet 210.5.51.32 netmask 255.255.255.255"
 > ifconfig_bce1_alias4="inet 210.5.51.33 netmask 255.255.255.255"
 > ifconfig_bce1_alias5="inet 210.5.51.34 netmask 255.255.255.255"
 > ifconfig_bce1_alias6="inet 210.5.51.42 netmask 255.255.255.255"

I didn't spot on first reading this that the first address is in a 
different subnet than all the others.  I'm not entirely sure whether 
that's relevant, or how, just pointing it out as being non-obvious, and 
suspecting one of the 210.5.51 subnet should show a broader netmask.

 > Relevant ipfw rules :
 > ipfw -q add 02012 allow tcp from any to 208.69.123.164 80 out via bce1 setup
 > keep-state
 > ipfw -q add 02012 allow tcp from any to 208.69.123.164 443 out via bce1 setup
 > keep-state

netstat -finet -rn (or -rna) please?  unclear where your default route 
goes, or how the 210.5.51 subnet is routed or its netmask, but assume 
that 208.69.123.164 is probably accessed via the default route ..

 > Interesting entries in /var/log/security :
 > Dec  1 16:42:25 <servername> kernel: ipfw: 9999 Deny TCP 210.5.50.5:49708
 > 208.69.123.164:80 out via bce1

Did that occur =after= the above rules were installed?  Just the one?  
Seems odd on face value, but without knowing what your other rules do.

 > What makes this interesting is that I can connect to that port via the
 > command line.

You mean like with 'telnet 208.69.123.164 80' ?  With 210.5.50.5 as 
source address?  tcpdump output may help understand or explain this.

 > It's the website that lives on 210.5.51.42 that is having problems. Why, if
 > the rule is valid enough for the command line is it having problems from an
 > aliased address?

Hang on; do you mean you're having a webserver on 210.5.51.42 trying to 
connect out to another webserver on 208.69.123.164 ?  If not, what?

I guess you have rules allowing inbound port 80 access to 210.5.51.42 ?

And that your upstream is routing 210.5.51.42/something to 210.5.50.5 ?

 > This MUST have something to do with the way ipfw is working with aliased
 > addresses but I'm blowed if I know what is wrong.

ipfw doesn't do anything different with any address in particular except 
when using the forward action.  ipfw certainly has no concept of primary 
or alias addresses, it just applies the addresses/masks you specify.

Nor does ipfw know or care (even when forwarding) whence the stack is 
next going to route outbound packets .. but netstat -rn will tell us.

cheers, Ian