kern/84802: kernel hangs with pf and route-to

Sergey Lapin slapinid at gmail.com
Thu Aug 11 14:50:05 GMT 2005


>Number:         84802
>Category:       kern
>Synopsis:       kernel hangs with pf and route-to
>Confidential:   no
>Severity:       critical
>Priority:       medium
>Responsible:    freebsd-bugs
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Thu Aug 11 14:50:04 GMT 2005
>Closed-Date:
>Last-Modified:
>Originator:     Sergey Lapin
>Release:        FreeBSD 6.0-BETA2 i386
>Organization:
>Environment:
System: FreeBSD int-s60-3.dart.spb 6.0-BETA2 FreeBSD 6.0-BETA2 #0: Wed Aug 3 08:22:24 UTC 2005 root at harlow.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC i386

>Description:
We are have 2 upstream providers. They give us 2 networks.
First, 1.1.1.0/24 is split by us into 2 parts by /25 mask - first part
is outer, and second (129-254) is behind our router (DMZ1)
second is 2.2.2.0/25 given by second ISP, is routed to our router,
and provider gives /30 p-to-p network 2.2.3.172/30 so 174 is our side and
173 is their's. So 2.2.2.0/25 is routed to 2.2.3.174 from their side.

Configuration:
(all addresses fake, 1.1.1.x - from ISP1, 2.2.2 - from ISP2)
# grep ifconfig /etc/rc.conf
ifconfig_xl0="inet 1.1.1.254 netmask 255.255.255.128"
ifconfig_xl0_alias0="inet 2.2.2.2 netmask 255.255.255.128"
ifconfig_xl1="inet 192.168.255.1 netmask 255.255.255.255"
ifconfig_vlan0="inet 1.1.1.3 netmask 255.255.255.0 vlan 1001 vlandev xl1 mtu 1496"
ifconfig_vlan1="inet 2.2.3.174 netmask 255.255.255.252 vlan 1004 vlandev xl1 mtu 1496"
# grep defaultrouter /etc/rc.conf
defaultrouter="1.1.1.1"
# cat /etc/pf.conf

#       $FreeBSD: src/etc/pf.conf,v 1.1.2.1 2004/09/17 18:27:14 mlaier Exp $
#       $OpenBSD: pf.conf,v 1.21 2003/09/02 20:38:44 david Exp $
#
# See pf.conf(5) and /usr/share/examples/pf for syntax and examples.
# Required order: options, normalization, queueing, translation, filtering.
# Macros and tables may be defined and used anywhere.
# Note that translation rules are first match while filter rules are last match.

ext_if1 = "vlan0"
ext_if2 = "vlan1"
dmz_if  = "xl0"
ext_gw1 = "1.1.1.1"
ext_gw2 = "2.2.3.173"

lan_net = "192.168.0.0/16"
dmz_net1 = "1.1.1.128/25"
dmz_net2 = "2.2.2.0/25"

table <our_nets> const { $dmz_net1, $dmz_net2, $lan_net }

set block-policy drop
set state-policy floating

#  Normalize all incoming streams
scrub in on $ext_if1
scrub in on $ext_if2

#################################################################################
#                             NAT
       #
#################################################################################

#  nat outgoing connections on each internet interface
nat on $ext_if1 from { $lan_net $dmz_net2 } to any -> ($ext_if1)
nat on $ext_if2 from { $lan_net $dmz_net1 } to any -> ($ext_if2)

#################################################################################
#  Block everything by default
       #
#################################################################################

#  default deny silently
block drop all

#  block IDENT notifying sender to prevent sendmail and the like from
#  wasting time waiting for timeout
block return in on { $ext_if1 $ext_if2 } proto { tcp, udp } to port = auth

block drop log on xl0 all

#################################################################################
#  Traffic to gateway itself
       #
#################################################################################

#  pass in quick any packets destined for the gateway itself
pass in quick on $dmz_if proto tcp from any to $dmz_if flags S/SA keep state
pass in quick on $dmz_if inet proto { udp, icmp } from any to $dmz_if keep state

#  pass multicast and IGMP traffic
pass quick on $dmz_if inet from any to 224.0.0.0/4 allow-opts keep state

pass quick on lo0

#################################################################################
#  Classify traffic from DMZ
       #
#################################################################################

#  pass traffic from DMZ to Internet
pass in on $dmz_if proto udp from $dmz_net1 to any port = 53 keep state tag DMZ_TO_EXT1
pass in on $dmz_if proto udp from $dmz_net2 to any port = 53 keep state tag DMZ_TO_EXT2
# Allow all outgoing connections from DMZ

pass in on $dmz_if inet proto tcp from $dmz_net1 to any flags S/SA keep state tag DMZ_TO_EXT1
pass in on $dmz_if inet proto { udp, icmp } from $dmz_net1 to any keep state tag DMZ_TO_EXT1

pass in on $dmz_if inet proto tcp from $dmz_net2 to any flags S/SA keep state tag DMZ_TO_EXT2
pass in on $dmz_if inet proto { udp, icmp } from $dmz_net2 to any keep state tag DMZ_TO_EXT2

#  Allow gateway to route between different networks on the DMZ

#  DMZ nets -> DMZ nets
pass in on $dmz_if inet proto tcp from { $dmz_net1, $dmz_net2 }  to { $dmz_net1, $dmz_net2 } flags S/SA keep state tag DMZ_TO_DMZ
pass in on $dmz_if inet from { $dmz_net1, $dmz_net2 } to { $dmz_net1, $dmz_net2 } keep state tag DMZ_TO_DMZ

#  DMZ nets -> LAN net
pass in on $dmz_if inet proto tcp from { $dmz_net1, $dmz_net2 } to $lan_net flags S/SA keep state tag DMZ_TO_LAN
pass in on $dmz_if inet from { $dmz_net1, $dmz_net2 } to $lan_net keep state tag DMZ_TO_LAN

#  LAN net -> DMZ nets
pass in on $dmz_if inet proto tcp from $lan_net to { $dmz_net1, $dmz_net2 } flags S/SA keep state tag LAN_TO_DMZ
pass in on $dmz_if inet from $lan_net to { $dmz_net1, $dmz_net2 } keep state tag LAN_TO_DMZ

#################################################################################
#  Allow classified traffic from DMZ
       #
#################################################################################

#  Allow incoming packets from DMZ one more time and route them appropriately
#  This must be done to IN packets because if we only do it for OUT
packets, it happens to late -
#  packet is routed appropriately, but NAT rule for wrong interface gets fired

pass in quick on $dmz_if route-to ($ext_if1 $ext_gw1) proto tcp tagged DMZ_TO_EXT1 flags S/SA modulate state
pass in quick on $dmz_if route-to ($ext_if1 $ext_gw1) proto { udp, icmp } tagged DMZ_TO_EXT1 keep state

pass in quick on $dmz_if route-to ($ext_if2 $ext_gw2) proto tcp tagged DMZ_TO_EXT2 flags S/SA modulate state
pass in quick on $dmz_if route-to ($ext_if2 $ext_gw2) proto { udp, icmp } tagged DMZ_TO_EXT2 keep state

#  Allow OUT traffic

pass out quick on $ext_if1 route-to ($ext_if2 $ext_gw2) proto tcp tagged DMZ_TO_EXT2 flags S/SA modulate state
pass out quick on $ext_if1 route-to ($ext_if2 $ext_gw2) proto { udp, icmp } tagged DMZ_TO_EXT2 keep state

pass out quick on $ext_if2 route-to ($ext_if1 $ext_gw1) proto tcp tagged DMZ_TO_EXT1 flags S/SA modulate state
pass out quick on $ext_if2 route-to ($ext_if1 $ext_gw1) proto { udp, icmp } tagged DMZ_TO_EXT1 keep state

#################################################################################
#  Clasify traffic from Internet to DMZ
       #
#################################################################################

# WHISKEY
pass in on vlan0 proto tcp from any to 1.1.1.144/32 port = 22 flags S/SA keep state tag EXT1_TO_DMZ
pass in on vlan1 proto tcp from any to 2.2.2.2/32 port = 22 flags S/SA keep state tag EXT2_TO_DMZ

#################################################################################
#  Allow classified traffic from Internet to DMZ
       #
#################################################################################

#  Pass to DMZ traffic already approved by earlier rules
#  and route replies to corresponding interface

#  EXT1
pass out quick on $dmz_if reply-to ($ext_if1 $ext_gw1) proto tcp tagged EXT1_TO_DMZ flags S/SA keep state
pass out quick on $dmz_if reply-to ($ext_if1 $ext_gw1) tagged EXT1_TO_DMZ keep state

#  EXT2
pass out quick on $dmz_if reply-to ($ext_if2 $ext_gw2) proto tcp tagged EXT2_TO_DMZ flags S/SA keep state
pass out quick on $dmz_if reply-to ($ext_if2 $ext_gw2) tagged EXT2_TO_DMZ keep state


#################################################################################
#  Other traffic
#
#################################################################################

#  general "pass out" rules for external interfaces
pass out on { $ext_if1, $ext_if2, $dmz_if } proto tcp from any to any flags S/SA modulate state
pass out on { $ext_if1, $ext_if2, $dmz_if } proto { udp, icmp } from any to any keep state

#  Zebra uses IGMP so let it work on DMZ interface
pass out on $dmz_if proto igmp from any to any allow-opts




Test case:
(done from Linix machine from 1.1.1.128/25)

tcpreplay -e 1.1.1.133:255.255.255.255 -i eth0 packet
(where packet is random captured _broadcast_ UDP packet using tcpdump -peni)

or

tcpreplay -e 1.1.1.133:10.2.2.2 -i eth0 packet
(where packet is random captured _broadcast_ UDP packet)

kills machine.
Machine hangs and doesn't react on keyboard, whatever.
Only reset helps.
More than that - machine sends that packet throught interface (that the same packet) continuously, like in loop, even while we're not sending it.
I don't need to say that the same ruleset works well on OpenBSD 3.7 (with s/modulate/keep/g)
Any ideas?
Thanks a lot!

>How-To-Repeat:
Just read description, find UDP packet with broadcast mac and try to sen
d it.
It doesn't reproduce inside VmWare.

>Fix:
Temp. workaround idea is disable broadcast UDP packets, but I don't
think it's too good.

>Release-Note:
>Audit-Trail:
>Unformatted:


More information about the freebsd-bugs mailing list