IPFW/natd/prob load balancing

[Patrick Tracanelli]

Hello,

I have a friend who wishes very much to do load balancing with IPFW and 
natd, and he doesnt want to do so using PF. Also, he needs arbitrary 
balancing, not round-robin, but instead to choose "X%" for one link and 
the rest to the other.

It cant be done for a number of reasons. First of, natd cant run 
attached to more than one interface. Instead, we need to run natd(8) 
instances, which are independent. If natd instances could work be aware 
of each other, maybe a hacker could add the balancing feature to it.

So I decided to give ipfw+prob a try, and try to help him out. I could 
get to some point, but got stucked, and cant help anymore. The 
psuedo-firewall (just a fragment of rules) I am using is:

#
fw="/sbin/ipfw"
ife="vr0"
ife2="vr1"
ife2_gw="201.86.82.1"
ife2_me="201.86.82.2"
rede_i="10.84.0.0/16"

#
#
#
$fw -f flush

$fw add prob 0.3 divert 8669 tcp from $rede_i to any out via $ife setup
$fw add prob 0.3 divert 8669 tcp from $rede_i to any out via $ife not setup
$fw add prob 0.3 divert 8669 { udp or icmp } from $rede_i to any out via 
$ife
$fw add fwd $ife2_gw all from $ife2_me to any out
$fw add divert 8669 all from any to any in via $ife2

$fw add divert 8668 tcp from $rede_i to any out via $ife setup
$fw add divert 8668 tcp from $rede_i to any out via $ife not setup
$fw add divert 8668 { udp or icmp } from $rede_i to any out via $ife setup
$fw add divert 8668 all from any to any in via $ife

And here the natd.conf:

instance default
unregistered_only yes
interface vr0
dynamic yes
use_sockets yes
same_ports yes
port 8668

instance link2
unregistered_only yes
interface vr1
dynamic yes
use_sockets yes
same_ports yes
port 8669

Why it wont work?

Because the "divert" stuff is per-packet, and not session aware. On the 
other hand I can not use keep-state in a divert rule. Also, I think the 
behavior of mixing keep-state and prob is not what we (I?) expect. I 
tried using "tag" and "not diverted" somewhere to identify packets that 
are already served from one link or the other, but no working idea 
occurred me.

Maybe any hacker or more experienced person can have a good suggestion?

I tried to help out on this question because for me it was a proof of 
concept that teorically (conceptually) it would be possible to balance 
this way. In fact it is, it is working partially.

But sometime, earlier or later the connection gets dropped (it is when 
prob does not apply, and the packet get diverted to another natd).

It doesnt work perfectly because of limitations of the tools or maybe I 
am missing a good idea.

So, dont bother answering to point all the reasons why it wont work ;) I 
am aware of all, also, I am aware of potential session issues (SSL 
sites, etc), when PF for example has a "sticky-address" solution.

The think is, maybe there is an easy solution that a hacker may think 
of, to allow natd or ipfw balancing outgoing sessions.