Problem with PF reply-to [SOLVED]

[Mario Lobo]

On Wednesday 13 July 2011 10:26:59 Mario Lobo wrote:
> Hi;
> 
> I have the following scenario.
> 
> FreeBSD 8.2-STABLE FreeBSD 8.2-STABLE #0: Thu May 19 19:53:59 BRT 2011
> i386
> 
> I want to be able to connect to any of the 2 external IPs this machine has.
> 
> ### pf.conf excerpt
> 
> ext_if1 = sis0 (1M link. default gateway)
> ext_if2 = rl0  (2M link)
> aln_if  = dc0  (Internal LAN)
> 
> ext_gw1 = A.A.A.A
> ext_gw2 = B.B.B.B
> 
> my_ext_ip1 = a.a.a.a
> my_ext_ip2 = b.b.b.b
> 
> 
>   nat on $ext_if1 from any to any -> $my_ext_ip1 port 1024:65535
>   nat on $ext_if2 from any to any -> $my_ext_ip2 port 1024:65535
> 
> 1) ----------------------------------------- # balance the load
> 
>   pass in log quick on $aln_if route-to ($ext_if2 $ext_gw2) from \
>   $aln_if:network to any flags S/SA keep state tag to_out probability 70%
> 
>   pass in log quick on $aln_if route-to ($ext_if1 $ext_gw1) from \
>   $aln_if:network to any flags S/SA keep state tag to_out
> 
> 2) ----------------------------------------- # allow ssh on ext_ifs
> 
> a)pass in log quick on $ext_if1 inet proto tcp from any to any port
> $SshPort \ flags S/SA modulate state (max 30, source-track rule,
> max-src-nodes 10,\ max-src-states 2, max-src-conn 2, max-src-conn-rate
> 2/60, overload <banned>)
> 
> b)pass in log quick on $ext_if2 reply-to ($ext_if2 $ext_gw2) inet proto tcp
> \ from any to any port $SshPort flags S/SA keep state (max 30,
> source-track \ rule, max-src-nodes 10, max-src-states 2, max-src-conn 2,
> max-src-conn-rate\ 2/60, overload <banned>) ( RULE 8 )
> 
> 
> [snip][snip]......
> 
> 
> 3) -----------------------------------------
> 
>   pass out quick on $ext_if1 route-to ($ext_if2 $ext_gw2) from $ext_if2 to
> any pass out quick on $ext_if2 route-to ($ext_if1 $ext_gw1) from $ext_if1
> to any
> 
>   Also tried:
> 
>   pass out quick on $ext_if1 route-to ($ext_if1 $ext_gw1) from $ext_if1 to
> any pass out quick on $ext_if2 route-to ($ext_if2 $ext_gw2) from $ext_if2
> to any
> 
>   block log all ( RULE 163 )
> 
> 
> ### end of pf.conf excerpt
> 
> 
> Everything under 1) works fine.
> 
> Under 2), a) works, b) not working.
> 
> When I try to connect to $SshPort through the 2M link (b.b.b.b). I connect
> to the server, but the return packet neither obeys the reply-to on rule
> b), nor matches any of the pass outs under 3), and goes straight to the
> block rule. as you can see bellow.
> 
> 
> [$]> tcpdump -n -e -ttt -i pflog0 host 187.113.99.63 (my home IP)
> 
> 
> Packet arrives and matches rule b)/8 and should create a state;
> 
> 00:00:00.000000 rule 8/0(match): pass in on rl0: 187.113.99.63.25806 >
> b.b.b.b.22: [|tcp]
> 
> but...
> 
> 00:00:00.000108 rule 163/0(match): block out on sis0: a.a.a.a.8947 >
> 187.113.99.63.25806: [|tcp]
> 00:00:03.000057 rule 163/0(match): block out on sis0: a.a.a.a.65060 >
> 187.113.99.63.25806: [|tcp]
> 00:00:03.199931 rule 163/0(match): block out on sis0: a.a.a.a..20213 >
> 187.113.99.63.25806: [|tcp]
> 00:00:03.199618 rule 163/0(match): block out on sis0: a.a.a.a..19748 >
> 187.113.99.63.25806: [|tcp]
> 00:00:03.200044 rule 163/0(match): block out on sis0: a.a.a.a..1600 >
> 187.113.99.63.25806: [|tcp]
> 00:00:03.199767 rule 163/0(match): block out on sis0: a.a.a.a..45513 >
> 187.113.99.63.25806: [|tcp]
> 00:00:06.205048 rule 163/0(match): block out on sis0: a.a.a.a..17925 >
> 187.113.99.63.25806: [|tcp]
> 
> it tries to go back to me on the wrong interface (sis0 and NOT rl0),
> wrong ip (a.a.a.a and NOT b.b.b.b), and from several wrong port numbers,
> not port 22.
> 
> Questions:
> 
> 1) sshd is listening on *.22. I know that the default gateway is not on rl0
> but isn't that what reply-to is supposed to beat? If I understood
> correctly, wasn't the reply-to supposed to make the packet go back
> throught the specified ($ext_if2 $ext_gw2)?
> 
> 2) Wasn't a state created when the pass rule b)/8 matched? if so, where is
> it?
> 
> Where am I doing wrong here?
> 
> Thanks for any hints.


Never mind !

I solved the problem after finding this very enlightening document:

http://www.mmacleod.ca/blog/2011/06/source-based-routing-with-freebsd-using-
multiple-routing-table/

I followed it and it all works beautifully now.

-- 
Mario Lobo
http://www.mallavoodoo.com.br
FreeBSD since 2.2.8 [not Pro-Audio.... YET!!] (99% winblows FREE)