Web Server supporting up to 4 WANs/Interfaces

[Julian Elischer]

On 12/16/10 10:32 PM, Jayster wrote:
> I am in need of immediate help and while I rarely post on boards or lists (I promise I'll start sharing more of my own wisdom in the future), this issue has me stumped like no other. First of all, I need multiple gateways. Yes, I understand there is no such thing as multiple "default" gateways and I have a solid grasp of networking in general... . I also have a reasonable grasp of FreeBSD, though I am no where near the expert of many. I actually come from the world of programming and I've only used Linux and Windows in the past.
>
> I am currently working on a big project that involves many sites and a custom appliance with multiple applications designed in-house. In fact, I might be the ONLY person in snowy Michigan hiring right now... . After an exhaustive investigation, I chose FreeBSD over all other OSs. It was not a light choice, but I've been very pleased despite having zero knowledge at the start of it. The main reasons were flexibility, licensing and most important, I'm still shocked how open the community is to answering questions in posts, instead of putting people down for having less knowledge than a rude poster. I have found an answer to the most obscure questions through very quick google searches...until now. Kudos to those who take the time...FreeBSD far outnumbers other OS communities. Please help me avoid this issue being the big letdown, possibly forcing us to leave FreeBSD. I like the OS way too much already.
>
> Now the issue. Without too much detail, my device has 4 GigE ports on it. Each will be attached to a routed network. There is NO routing required between networks inside the box (not a router or firewall) and in fact, it CANNOT be allowed to happen because of security. Instead, each WAN port needs access to this box, but nothing beyond. The access consists of a Web Server, though several other Ports are required, such as SNMP Traps, Syslog, etc. Getting to the box is easy, routers do all the work. The issue is getting traffic back through the same interface it came in on and through the same router gateway. As we all know, only 1 gateway can be assigned in FreeBSD, unlike other flavors of Linux. Even the ones who don't offer single line gateway support can use IPTables to accomplish this task. But IPTables is not supported in FreeBSD. Not a bad thing as long as comparable solutions exist.
>
> Setting up static routes is not the solution. The problem with it is that multiple sites which will have this box will not have access to the next hop info from the gateway (the next hop gateway and subnet on the other side of the router). So I cannot use static routes.
>
> PFSense appears to support this (though not tested by me). I REALLY do not want to go that route. We have invested 3 months into adding many apps to the FreeBSD we have. PFSense is a custom FreeBSD kernel with many changes. Many message boards claim it breaks many Ports and changes other behaviors. Even if it didn't, we are under deadline and moving everything over to a new FreeBSD Version and then extensively testing everything repeatedly again would be a nightmare. I am interested in experiences with it if it becomes the last resort, though.
>
> I have tried both PF and IPFW. Different posts around the web claim Multiple Gateway solutions using both of them. I have tried each of the recommended setups, but had no luck. If you read the last responses to each of those posts, others also state they could not duplicate what is claimed, as well. PF looks the most promising. It has "if-bound", which is supposed to keep interface traffic on the same interface. That is a good first step. But pointing it to the gateway on that interface is still an issue. Please HELP!!! I haven't slept in days and I've been stuck for a week now!!! This is our last showstopper.
>

if you are running on freeBSD 8 or newer you have two solutions open 
to you (maybe 3).

* Firstly, you can assign a completely different routing table to each 
socket so that
packets from one socket only see things through the perspective of one 
routing
table but packets from another socket behave according to the rules of 
a completely
different routing table. alternatively you can assign a different 
routing table to a
process and its descendants.
you can also use ipfw fwd to remap ports and addresses, in conjunction 
with the
different routing tables.  Routing tables are sometimes called FIBs 
(Forwarding Information Bases)

man setfib(1) and setfib(2) for more details.
Also see the setfib socket option in setsockopt(2).

There are details that are still open for development (like IPV6 
support) but
it sounds like it will do what you want.

* The second option is the new jail support.

while setfib and friends can easily allow a single process to act 
differently on a socket by socket basis
the new jail facilities allow you to take multiple interfaces and 
assign them to different jails,
and each jail can be given a completely different routing table or in 
fact a completely different SET of routing tables.

man jail   and man ifconfig            (ifconfig vnet)

* As a poor third contender you can do really funky things with the 
ipfw 'fwd' command.

julian

(let me know offline a bit more about what you want and maybe I can be 
a bit more specific about how to do it.)



> Jay
> _______________________________________________
> freebsd-net at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-net
> To unsubscribe, send any mail to "freebsd-net-unsubscribe at freebsd.org"
>