kern/144187: deadlock using multiple ipfw nat and multiple limit statements

Dmitriy Demidov dima_bsd at inbox.lv
Sun Feb 21 19:10:02 UTC 2010


>Number:         144187
>Category:       kern
>Synopsis:       deadlock using multiple ipfw nat and multiple limit statements
>Confidential:   no
>Severity:       critical
>Priority:       high
>Responsible:    freebsd-bugs
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Sun Feb 21 19:10:01 UTC 2010
>Closed-Date:
>Last-Modified:
>Originator:     Dmitriy Demidov
>Release:        7.3-PRERELEASE
>Organization:
>Environment:
FreeBSD evo.local.home 7.3-PRERELEASE FreeBSD 7.3-PRERELEASE #0: Sat Feb 20 22:57:09 EET 2010     root at evo.local.home:/usr/obj/usr/src/sys/STABLE  i386
>Description:
I met one problem with multiple ipfw nat instances and multiple limit rules. If I configure my ipfw as showed below and make a try to connect to 192.168.1.2 from 192.168.1.1 using ssh, then system just die without any error messages or panics. It do not reboots but just stay in deadlock forever - I am unable to use console or ssh/ping it anymore, and only way to get it back is a reset or power buttons.

Problem disappears only then I remove "limit src-addr 80" from first, or from second nat instances.

System is a nat router what have 3 network interfaces - fxp0 is connected to inner LAN (172.16.1.0/24) and each of rl adapters is connected to different upstreem ISP. There is no any setfib at this moment. Default gateway (192.168.1.1) is on rl0 side.


Customized GENERIC kernel:
===
#cpu            I486_CPU
#cpu            I586_CPU
cpu             I686_CPU
ident           STABLE

options         IPFIREWALL
options         IPFIREWALL_NAT
options         LIBALIAS
options         ROUTETABLES=2
options         DUMMYNET
options         HZ=1000

makeoptions     DEBUG=-g

options         KDB
options         DDB
options         GDB
options         INVARIANTS
options         INVARIANT_SUPPORT
options         WITNESS
options         DEBUG_LOCKS
options         DEBUG_VFS_LOCKS
options         DIAGNOSTIC
##
===

sysctl:
===
sysctl -a | grep one_pass
net.inet.ip.fw.one_pass: 0
===

ipfw configuration:
===
add 1040 allow ip from any to any via fxp0

nat 1 config if rl0 reset same_ports deny_in redirect_port tcp 192.168.1.2:22 22
nat 2 config if rl1 reset same_ports deny_in

add 10130 nat 1 tcp from any to any out xmit rl0 limit src-addr 80
add 10131 allow tcp from any to any out xmit rl0
add 10140 nat 1 ip from any to any out xmit rl0
add 10141 allow ip from any to any out xmit rl0

add 20130 nat 2 tcp from any to any out xmit rl1 limit src-addr 80
add 20131 allow tcp from any to any out xmit rl1
add 20140 nat 2 ip from any to any out xmit rl1
add 20141 allow ip from any to any out xmit rl1

add 20150 nat 1 ip from any to any in recv rl0
add 20151 nat 2 ip from any to any in recv rl1

add 30160 allow ip from any to any
add 65534 deny ip from any to any
===

ipfw show:
===
00100 0  0 allow ip from any to any via lo0
00200 0  0 deny ip from any to 127.0.0.0/8
00300 0  0 deny ip from 127.0.0.0/8 to any
01040 0  0 allow ip from any to any via fxp0
10130 1 40 nat 1 tcp from any to any out xmit rl0 limit src-addr 80
10131 1 40 allow tcp from any to any out xmit rl0
10140 0  0 nat 1 ip from any to any out xmit rl0
10141 0  0 allow ip from any to any out xmit rl0
20130 0  0 nat 2 tcp from any to any out xmit rl1 limit src-addr 80
20131 0  0 allow tcp from any to any out xmit rl1
20140 0  0 nat 2 ip from any to any out xmit rl1
20141 0  0 allow ip from any to any out xmit rl1
20150 1 52 nat 1 ip from any to any in recv rl0
20151 0  0 nat 2 ip from any to any in recv rl1
30160 1 52 allow ip from any to any
65534 0  0 deny ip from any to any
65535 0  0 deny ip from any to any
===

ifconfig:
===
evo# ifconfig
rl0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=8<VLAN_MTU>
        ether 00:30:4f:11:11:11
        inet 192.168.1.2 netmask 0xffffff00 broadcast 192.168.1.255
        media: Ethernet autoselect (100baseTX <full-duplex>)
        status: active
fxp0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=2009<RXCSUM,VLAN_MTU,WOL_MAGIC>
        ether 00:0b:cd:22:22:22
        inet 172.16.1.1 netmask 0xffffff00 broadcast 172.16.1.255
        media: Ethernet autoselect (none)
        status: no carrier
rl1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=8<VLAN_MTU>
        ether 00:0c:46:33:33:33
        inet 192.168.2.2 netmask 0xffffff00 broadcast 192.168.2.255
        media: Ethernet autoselect (100baseTX <full-duplex>)
        status: active
plip0: flags=108810<POINTOPOINT,SIMPLEX,MULTICAST,NEEDSGIANT> metric 0 mtu 1500
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x5
        inet6 ::1 prefixlen 128
        inet 127.0.0.1 netmask 0xff000000

===

===
evo# setfib 0 netstat -rn
Routing tables

Internet:
Destination        Gateway            Flags    Refs      Use  Netif Expire
default            192.168.1.1        UGS         0        0    rl0
127.0.0.1          127.0.0.1          UH          0        0    lo0
172.16.1.0/24      link#2             UC          0        0   fxp0
192.168.1.0/24     link#1             UC          0        0    rl0
192.168.1.1        00:90:27:2c:12:12  UHLW        1       20    rl0   1193
192.168.2.0/24     link#3             UC          0        0    rl1

Internet6:
Destination                       Gateway                       Flags      Netif Expire
::1                               ::1                           UHL         lo0
fe80::%lo0/64                     fe80::1%lo0                   U           lo0
fe80::1%lo0                       link#5                        UHL         lo0
ff01:5::/32                       fe80::1%lo0                   UC          lo0
ff02::%lo0/32                     fe80::1%lo0                   UC          lo0

evo# setfib 1 netstat -rn
Routing tables

Internet:
Destination        Gateway            Flags    Refs      Use  Netif Expire
127.0.0.1          127.0.0.1          UH          0        0    lo0
172.16.1.0/24      link#2             UC          0        0   fxp0
192.168.1.0/24     link#1             UC          0        0    rl0
192.168.1.1        00:90:27:2c:12:12  UHLW        1        0    rl0   1190
192.168.2.0/24     link#3             UC          0        0    rl1

Internet6:
Destination                       Gateway                       Flags      Netif Expire
::1                               ::1                           UHL         lo0
fe80::%lo0/64                     fe80::1%lo0                   U           lo0
fe80::1%lo0                       link#5                        UHL         lo0
ff01:5::/32                       fe80::1%lo0                   UC          lo0
ff02::%lo0/32                     fe80::1%lo0                   UC          lo0

===
>How-To-Repeat:
Configure kernel options and ipfw rules as it mentioned in Description and try to connect to system via ssh (or just telnet to port 22).
>Fix:
Do not use multiple limit statements.

>Release-Note:
>Audit-Trail:
>Unformatted:


More information about the freebsd-bugs mailing list