Best practices for service provider?

[David Pierron]

Soren Worach on 11/18/2005 7:19 PM wrote:

>On Friday 18 November 2005 18:26, Danny Fullerrton wrote:
>  
>
>>David Pierron wrote:
>>    
>>
>>>This is a loaded question so please bear with me.   I could really use
>>>the advice/help.
>>>
>>>I am coming from a FreeBSD 4.9 IPLess IPFW Bridging Firewall ...  I
>>>had followed the directions from the FreeBSD Handbook ...  Recently it
>>>crashed, so I had to rebuild it, uhm ... quickly ...
>>>
>>>This time I decided to include a 3rd NIC so that I could get the
>>>nightly emails and pay a bit better attention to its status ...  It is
>>>working, but giving me some errors about arp: xx:xx:xx:xx:xx:xx is
>>>using my IP address my.c.class.xx!  I have been scouring the Internet
>>>for information, and I decided to give PF a try ...  I installed
>>>OpenBSD 3.8 but didn't like its CLI interface ...  Not that I use a
>>>GUI, I don't ... I just hop around much better on FreeBSD ...
>>>
>>>I drew a picture of what I am envisioning as a firewall solution for
>>>me here:
>>>http://www.davidpierron.com/img/net-map.jpg
>>>
>>>I installed FreeBSD 6.0 and cvsup'd ports and src ... put the
>>>following into GENERIC:
>>>
>>># to allow bridge support
>>>device if_bridge
>>>
>>>#PF
>>>device    pf
>>>device    pflog
>>>device    pfsync
>>>
>>>#ALTQ
>>>options         ALTQ
>>>options         ALTQ_CBQ        # Class Bases Queuing (CBQ)
>>>options         ALTQ_RED        # Random Early Detection (RED)
>>>options         ALTQ_RIO        # RED In/Out
>>>options         ALTQ_HFSC       # Hierarchical Packet Scheduler (HFSC)
>>>options         ALTQ_PRIQ       # Priority Queuing (PRIQ)
>>>#options         ALTQ_NOPCC      # Required for SMP build
>>>
>>># other stuff
>>>options IPSTEALTH
>>>options HZ=1000
>>>
>>>I put the following into rc.conf:
>>>
>>>defaultrouter="my.c.class.1"
>>>hostname="firewall.foo.org"
>>>ifconfig_xl0="inet my.c.class.2  netmask 255.255.255.0"
>>>usbd_enable="NO"
>>>sendmail_enable="NO"
>>>
>>>cloned_interfaces="bridge0"          # create a bridge
>>>ifconfig_bridge0="addm rl0 addm rl1" # set bridge to use particular NICs
>>>#gateway_enable="YES"
>>>
>>>pf_enable="YES"                      # Enable PF (load module if
>>>required)
>>>pf_rules="/etc/pf.conf"              # rules definition file for pf
>>>pf_flags=""                          # additional flags for pfctl startup
>>>pflog_enable="YES"                   # start pflogd(8)
>>>pflog_logfile="/var/log/pflog"       # where pflogd should store the
>>>logfile
>>>pflog_flags=""                       # additional flags for pflogd
>>>startup
>>>
>>>.. and into sysctl.conf:
>>>
>>>net.link.bridge.pfil_bridge=1    # enables packet filtering on bridge
>>>net.link.bridge.pfil_member=1    # enables packet filtering on in and
>>>out interfaces
>>>#net.inet.ip.forwarding=1         # instead of gateway_enable in rc.conf?
>>>
>>>I am running into one of two things ... Trying to find information
>>>that isn't widely available yet, or trying to figure this out from old
>>>posts that don't apply anymore ...  The other thing going against me
>>>is that I haven't seen anything that resembles my setup ...  I am not
>>>running any NAT ...  I am using real world routable IP addresses ... I
>>>am assuming I need a 3rd NIC to be separate from the firewall ...
>>>      
>>>
>>You can use firewalled interface or bridge interface as normal interface
>>too. It's only depending on your config. You'll find lots of stuff on
>>google refering to a setup like yours but when searching for OpenBSD stuff.
>>    
>>
I have been using Google and searching ... I have not been successful in 
finding a HOW-TO or something similiar to help me configure this FreeBSD 
6.0 machine the way it ought to be configured ...  Many sites and 
spiders of mailing lists are outdated ...  As stated above, I want to 
use FreeBSD for this solution ...

>>>From my recent readings of this lists archives, it doesn't seem that I
>>>would want to run a bridge ...  It won't allow me to keep state ...
>>>If this is the case, how do I not assign the network cards that will
>>>be doing the filtering no ip address?  I tried some interesting
>>>combinations with ifconfig in rc.conf, but they didn't work ...  When
>>>I thought everything was up and running correctly, I put this box
>>>between my router and switch but traffic didn't flow ... I could ping
>>>internally, but could not ping the router's address which is the
>>>gateway (x.x.x.1) ...  I assumed that the internal pinging was working
>>>on the 3rd NIC with the real IP address ...
>>>      
>>>
>>Statefull mode is working in bridge mode using OpenBSD PF. But I dont
>>known if it's presently the case with the FreeBSD implementation.
>>    
>>
>
>it _is_ the case, pf supports statefull with bridging. I'm using 6.0 since 
>betaX on a couple of setups like this.
>  
>
I found messages in this archive only months old that suggest that 
although state is displayed that it may not be reporting correctly ...  
these messages were from 12/2004 and Jan/2005, and looking at them 
again, it's possible that they weren't even talking about if_bridge ...

>>>My question is, can I use two NICs for PF to do firewalling on to put
>>>between the router and the switch and then plug the 3rd NIC in and
>>>have it act as a separate interface on the box, or should I simply use
>>>2 NICs and assign them real IP addresses ...  If I do that, will
>>>IPSTEALTH compiled into the kernel not show the presence of the
>>>filtering?
>>>      
>>>
>>As I said, you could use this kind of setup (3 card to keep it simple
>>logic) or ,while using 2 interface in bridge mode, use 1 of them with an
>>internal ip address (bridge and standard).
>>
>>    
>>
>>>I think I have successfully confused myself with redundant or old
>>>information out there on the 'net, so again ... any suggestions or
>>>advice on what I am trying to accomplish would be greatly appreciated.
>>>      
>>>
>
>please post your pf.conf.
>  
>
Whoa ... we're not even there yet ... I am trying to get the hardware 
configured ... I am not clear as to the parameters required for the 
bridge or the options to allow IP Forwarding across the bridge and 
keeping the 3rd NIC separate ...  I set up a simple pf.conf to block all 
traffic:

scrub in all
block out log on $ext_if all
block in  log on $ext_if all

I saw no activity logged at all when I attached cables from the router 
and then to the switch ...

>>>Thank you for reading,
>>>David Pierron
>>>_______________________________________________
>>>      
>>>
>>You should begin by playing with Packet Filter while being in bridge
>>mode and gradually including feature like the management ip/interface
>>before going to far and not understanding.
>>
>>Danny Fullerton
>>    
>>
I think my initial problem when installing the 3 NICs and giving one an 
IP address is that they all use the default gateway ... Do I need to 
install the gateway just to the 3rd NIC somehow? (which I would call the 
management NIC) ... Should I remove "defaultrouter="x.x.x.1"" from 
rc.conf?  I would have thought the bridge would live in his own space ...

The bridge just needs to filter packets not caring about its own IP 
addresses ...  I would be able to deny or throttle by destination IP, 
but the bridge itself should see traffic coming in, filter it based on 
the rules, and then pass it on if okay or drop it if not okay ...  The 
outside world wouldn't know that there was an extra hardware appliance 
hop to their destination ...

The assumption in using 3 NICs is that FreeBSD will run an IPLess 
stateful packet filter on the 2 NIC bridge, the 3rd NIC's traffic will 
eventually travel across that bridge as shown in the diagram I drew ...  
This has to be possible, but there must be some trick to it that I 
haven't grasped ... Not many setups or HOWTOs explain this sort of setup 
or idea ...

Maybe I should have asked one question at a time?  I just thought this 
was all encompassing ... the hardware setup supporting the PF machine ...

David Pierron