udp - weird behavior of reply-to

[Marek Zarychta]

On Sun, Jan 08, 2017 at 07:08:10PM +0100, Kristof Provost wrote:
> On 8 Jan 2017, at 15:55, Marek Zarychta wrote:
> The problem description doesn’t ring any bells with me, but I’m also 
> not sure
> I’ve fully understood it.  Can you document a minimal reproduction 
> scenario,
> with a pf.conf and perhaps network captures documenting the problem?
> 

Network captures taken with tcpdump are quite simple:

1st msg from client
20:20:38.726593 IP 62.133.x.y.38315 > 88.199.x.y.1197: UDP, length 21
2nd msg from client
20:20:45.105679 IP 62.133.x.y.38315 > 88.199.x.y.1197: UDP, length 21
20:20:45.106680 IP 88.199.x.y > 62.133.x.y: ICMP 88.199.x.y udp port
1197 unreachable, length 36
1st reply from service: 
20:21:11.191630 IP 88.199.y.z.1197 > 62.133.x.y.38315: UDP, length 24
2nd reply from service: 
20:21:44.838787 IP 88.199.y.z.1197 > 62.133.x.y.38315: UDP, length 37

Only one UDP datagram passes the firewall from client to server, the
rest is bounced. All the replies are sent via wrong interface.
When I start service with another fib, where the interface has default
gateway in scope, communication goes fine. It could be still possible to
run two instances of service, but this is not what reply-to was intended
for.

By the way, negotiation of TCP connection via second interface goes
sucessful:
20:23:52.143832 IP 62.133.x.y.42426 > 88.199.105.83.22: Flags [S], seq
3881242448, win 29200, options [mss 1412,sackOK,TS val 57770500 ecr
0,nop,wscale 7], length 0
20:23:52.143927 IP 88.199.x.y.22 > 62.133.x.y.42426: Flags [S.], seq
430799235, ack 3881242449, win 65535, options [mss 1412,nop,wscale
9,sackOK,TS val 615314394 ecr 57770500], length 0
20:23:52.163432 IP 62.133.x.y.42426 > 88.199.x.y.22: Flags [.], ack 1,
win 229, options [nop,nop,TS val 57770505 ecr 615314394], length 0

The minimal pf.conf for use in reproduction scenario is attached. 

-------------- next part --------------
ext_if		= "em0" 	#  em0 is parent interface of vlan2
ext_if_2	= "vlan2"

ip_gw_1		= "88.199.p.q" 	# ip_gw_1 is default gateway 
ip_gw_2		= "88.199.r.s" 	# ip_gw_2 is default gw for fib 1

# uslugi
tcp_services       = "{ 22, 50000:55000 }"
udp_services       = "{ 1194:1199 }"

TCP_OPTIONS 	= "flags S/SA keep state"
UDP_OPTIONS 	= "keep state"

set block-policy return
set loginterface $ext_if
set skip on { lo, tun }

scrub in on {$ext_if, $ext_if_2} all

# ----
# ICMP
# ----
  pass out quick on { $ext_if, $ext_if_2 } inet proto icmp all \
       icmp-type 8 code 0 keep state 

  pass in quick on $ext_if inet proto icmp all \
       icmp-type 8 code 0 keep state 

  pass in quick on $ext_if_2 reply-to ( $ext_if_2 $ip_gw_2 ) \
       inet proto icmp all \
       icmp-type 8 code 0 keep state

# ---
# UDP
# ---
  pass in quick on $ext_if inet proto udp \
       from any \
       to ($ext_if:0) port $udp_services \
       $UDP_OPTIONS 
                       
  pass in quick on $ext_if_2 \
       reply-to ( $ext_if_2 $ip_gw_2 ) \
       inet proto udp \
       from any \
       to ($ext_if_2:0) port $udp_services \
       $UDP_OPTIONS 

  pass out quick on {$ext_if, $ext_if_2} proto udp \
       all \
       $UDP_OPTIONS 

# ---
# TCP
# ---
  pass in quick on $ext_if inet proto tcp \
       from any \
       to ($ext_if:0) port $tcp_services \
       $TCP_OPTIONS  

  pass in quick on $ext_if_2 \
       reply-to ( $ext_if_2 $ip_gw_2 ) \
       inet proto tcp \
       from any \
       to ($ext_if_2:0) port $tcp_services \
       $TCP_OPTIONS  

  pass out quick on {$ext_if, $ext_if_2} proto tcp \
       all \
       $TCP_OPTIONS 

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: not available
URL: <http://lists.freebsd.org/pipermail/freebsd-pf/attachments/20170108/8093801b/attachment.sig>