kern/164400: immediate crash after the start of ipsec processing
Eugene M. Zheganin
eugene at zhegan.in
Mon Jan 23 08:10:15 UTC 2012
>Number: 164400
>Category: kern
>Synopsis: immediate crash after the start of ipsec processing
>Confidential: no
>Severity: critical
>Priority: high
>Responsible: freebsd-bugs
>State: open
>Quarter:
>Keywords:
>Date-Required:
>Class: sw-bug
>Submitter-Id: current-users
>Arrival-Date: Mon Jan 23 08:10:13 UTC 2012
>Closed-Date:
>Last-Modified:
>Originator: Eugene M. Zheganin
>Release: 9.0-RELEASE
>Organization:
RealService LLC
>Environment:
FreeBSD 9.0-RELEASE FreeBSD 9.0-RELEASE #1: Sun Jan 22 21:59:51 MSK 2012 emz at moscow-alpha:/usr/obj/usr/src/sys/MOSCOW amd64
>Description:
There's a HA-cluster of 2 freebsd running ipsec+gre and a butch of tunnels to branch offices with OSPF. Both were running 8.2-RELEASE and/or different versions of 8.2-STABLE. Since this setup wasn't that stable I was constantly upgrading one of the nodes to a recent -STABLE. After a December -STABLE first node was crashing every hour, so I decided to test it with the memtest86+ (4.20). One week of running memtest gave no errors. So I upgraded to 9.0 and built a debug kernel with WITNESS/INVARIANTS and stuff.
OSPF is served by quagga.
ISAKMP is served by security/ipsec-tools.
Now I have an immediate crash after a carp(4) with public address for gre tunnels and ipsec processing is switching to MASTER. Without it it runs fine.
The crash is reproduceable (at least I tested 3 times and got 3 craches). BTs are identical ( I compared each screen first and last line, so only first set of screens is referenced here).
This host has an ipkvm (and YES, I can give access to it if needed, it lives on a public address, you will need a working java browser plugin to use it), so here are the screens of this trap happening:
http://tech.norma.perm.ru/files/screen01.jpeg
http://tech.norma.perm.ru/files/screen02.jpeg
http://tech.norma.perm.ru/files/screen03.jpeg
http://tech.norma.perm.ru/files/screen04.jpeg
http://tech.norma.perm.ru/files/screen05.jpeg
Furthermore, when running with WITNESS on this node produces the following output immediately after loading a set of pf rules:
http://tech.hq.norma.perm.ru/files/lor.txt
This output is always the same too.
ipsec.conf:
[emz@:~]> cat /etc/ipsec.conf
spdflush;
#
# Moscow, Schelkovskoye, Megaton
#
spdadd 94.159.37.114 89.250.210.69 gre -P out ipsec esp/transport/94.159.37.114-89.250.210.69/require ah/transport/94.159.37.114-89.250.210.69/require;
spdadd 89.250.210.69 94.159.37.114 gre -P in ipsec esp/transport/89.250.210.69-94.159.37.114/require ah/transport/89.250.210.69-94.159.37.114/require;
#
# Moscow, Pervomayskaya, MGTS
#
spdadd 109.252.242.9 94.159.37.114 gre -P in ipsec esp/transport/109.252.242.9-94.159.37.114/require ah/transport/109.252.242.9-94.159.37.114/require;
spdadd 94.159.37.114 109.252.242.9 gre -P out ipsec esp/transport/94.159.37.114-109.252.242.9/require ah/transport/94.159.37.114-109.252.242.9/require;
#
# Moscow, Privolnaya, 70
#
spdadd 82.142.171.58 94.159.37.114 gre -P in ipsec esp/transport/82.142.171.58-94.159.37.114/require ah/transport/82.142.171.58-94.159.37.114/require;
spdadd 94.159.37.114 82.142.171.58 gre -P out ipsec esp/transport/94.159.37.114-82.142.171.58/require ah/transport/94.159.37.114-82.142.171.58/require;
#
# Moscow, Lermontovsky, 2
#
spdadd 79.120.78.118 94.159.37.114 gre -P in ipsec esp/transport/79.120.78.118-94.159.37.114/require ah/transport/79.120.78.118-94.159.37.114/require;
spdadd 94.159.37.114 79.120.78.118 gre -P out ipsec esp/transport/94.159.37.114-79.120.78.118/require ah/transport/94.159.37.114-79.120.78.118/require;
#
# Moscow, Tashkentskaya 12-20
#
spdadd 79.120.80.66 94.159.37.114 gre -P in ipsec esp/transport/79.120.80.66-94.159.37.114/require ah/transport/79.120.80.66-94.159.37.114/require;
spdadd 94.159.37.114 79.120.80.66 gre -P out ipsec esp/transport/94.159.37.114-79.120.80.66/require ah/transport/94.159.37.114-79.120.80.66/require;
#
# Moscow, Baykalskaya 50/7
#
spdadd 213.33.223.158 94.159.37.114 gre -P in ipsec esp/transport/213.33.223.158-94.159.37.114/require ah/transport/213.33.223.158-94.159.37.114/require;
spdadd 94.159.37.114 213.33.223.158 gre -P out ipsec esp/transport/94.159.37.114-213.33.223.158/require ah/transport/94.159.37.114-213.33.223.158/require;
#
# Moscow, Bratyslavskaya 15-1
#
#spdadd 212.46.203.106 94.159.37.114 gre -P in ipsec esp/transport/212.46.203.106-94.159.37.114/require ah/transport/212.46.203.106-94.159.37.114/require;
#spdadd 94.159.37.114 212.46.203.106 gre -P out ipsec esp/transport/94.159.37.114-212.46.203.106/require ah/transport/94.159.37.114-212.46.203.106/require;
#add 212.46.203.106 94.159.37.114 esp 0x10001 -m transport -E des-cbc 0xffffffffffffffff;
#add 94.159.37.114 212.46.203.106 esp 0x10002 -m transport -E des-cbc 0xffffffffffffffff;
#add 212.46.203.106 94.159.37.114 ah 0x10003 -m transport -A keyed-md5 "xxxxxxxxxxxxxxxx";
#add 94.159.37.114 212.46.203.106 ah 0x10004 -m transport -A keyed-md5 "xxxxxxxxxxxxxxxx";
racoon.conf:
[emz@:~]# cat /usr/local/etc/racoon/racoon.conf
path pre_shared_key "/usr/local/etc//racoon/psk.txt";
# "padding" defines some padding parameters. You should not touch these.
padding {
maximum_length 20;<># maximum padding length.
randomize off; # enable randomize length.
strict_check off; # enable strict check.
exclusive_tail off; # extract last one octet.
}
# if no listen directive is specified, racoon will listen on all
# available interface addresses.
listen {
isakmp 94.159.37.114 [500];
strict_address; # requires that all addresses must be bound.
}
# Specify various default timers.
timer {
# These value can be changed per remote node.
counter 5; # maximum trying count to send.
interval 20 sec; # maximum interval to resend.
persend 1; # the number of packets per send.
# maximum time to wait for completing each phase.
phase1 30 sec;
phase2 15 sec;
}
#
# Kosm65, wizard, PiC
#
remote 89.250.210.69 {
exchange_mode main;
lifetime time 1 hour;
my_identifier address 94.159.37.114;
peers_identifier address 89.250.210.69;
passive off;
proposal_check obey;
dpd_delay 20;
proposal {
encryption_algorithm des;
hash_algorithm md5;
authentication_method pre_shared_key;
dh_group modp768;
}
}
#
# Moscow, Permovayskaya10/5, MGTS
#
remote 109.252.242.9 {
exchange_mode main;
lifetime time 1 hour;
my_identifier address 94.159.37.114;
peers_identifier address 109.252.242.9;
passive off;
proposal_check obey;
dpd_delay 20;
proposal {
encryption_algorithm des;
hash_algorithm md5;
authentication_method pre_shared_key;
dh_group modp768;
}
}
#
# Moscow, Privolnaya, 70
#
remote 82.142.171.58 {
exchange_mode main;
lifetime time 1 hour;
my_identifier address 94.159.37.114;
peers_identifier address 82.142.171.58;
passive off;
proposal_check obey;
dpd_delay 20;
proposal {
encryption_algorithm des;
hash_algorithm md5;
authentication_method pre_shared_key;
dh_group modp768;
}
}
#
# Moscow, Lermontovsky, 2
#
remote 79.120.78.118 {
exchange_mode main;
lifetime time 1 hour;
my_identifier address 94.159.37.114;
peers_identifier address 79.120.78.118;
passive off;
proposal_check obey;
dpd_delay 20;
proposal {
encryption_algorithm des;
hash_algorithm md5;
authentication_method pre_shared_key;
dh_group modp768;
}
}
#
# Moscow, Baykalskaya 50/7
#
remote 213.33.223.158 {
exchange_mode main;
lifetime time 1 hour;
my_identifier address 94.159.37.114;
peers_identifier address 213.33.223.158;
passive off;
proposal_check obey;
dpd_delay 20;
proposal {
encryption_algorithm des;
hash_algorithm md5;
authentication_method pre_shared_key;
dh_group modp768;
}
}
#
# Moscow, Tashkentskaya 12-20
#
remote 79.120.80.66 {
exchange_mode main;
lifetime time 1 hour;
my_identifier address 94.159.37.114;
peers_identifier address 79.120.80.66;
passive off;
proposal_check obey;
dpd_delay 20;
proposal {
encryption_algorithm des;
hash_algorithm md5;
authentication_method pre_shared_key;
dh_group modp768;
}
}
#
# Moscow, Bratyslavskaya 15-1
#
#remote 212.46.203.106 {
# exchange_mode main;
# lifetime time 1 hour;
# my_identifier address 94.159.37.114;
# peers_identifier address 212.46.203.106;
# passive off;
# proposal_check obey;
# dpd_delay 20;
# proposal {
# encryption_algorithm des;
# hash_algorithm md5;
# authentication_method pre_shared_key;
# dh_group modp768;
# }
#}
#
# kosm65, wizard, PiC
#
sainfo address 89.250.210.69 [500] 47 address 94.159.37.114 [500] 47 {
pfs_group modp768;
lifetime time 60 min;
encryption_algorithm blowfish;
authentication_algorithm hmac_md5;
compression_algorithm deflate;
}
sainfo address 94.159.37.114 [500] 47 address 89.250.210.69 [500] 47 {
pfs_group modp768;
lifetime time 60 min;
encryption_algorithm blowfish;
authentication_algorithm hmac_md5;
compression_algorithm deflate;
}
#
# Moscow, Pervomayskaya, 10/5, MGTS
#
sainfo address 94.159.37.114 [500] 47 address 109.252.242.9 [500] 47 {
pfs_group modp768;
lifetime time 60 min;
encryption_algorithm des;
authentication_algorithm hmac_sha1, non_auth;
compression_algorithm deflate;
}
sainfo address 109.252.242.9 [500] 47 address 94.159.37.114 [500] 47 {
pfs_group modp768;
lifetime time 60 min;
encryption_algorithm des;
authentication_algorithm hmac_sha1, non_auth;
compression_algorithm deflate;
}
#
# Moscow, Privolnaya, 70
#
sainfo address 94.159.37.114 [500] 47 address 82.142.171.58 [500] 47 {
pfs_group modp768;
lifetime time 60 min;
encryption_algorithm des;
authentication_algorithm hmac_sha1, non_auth;
compression_algorithm deflate;
}
sainfo address 82.142.171.58 [500] 47 address 94.159.37.114 [500] 47 {
pfs_group modp768;
lifetime time 60 min;
encryption_algorithm des;
authentication_algorithm hmac_sha1, non_auth;
compression_algorithm deflate;
}
#
# Moscow, Lermontovsky, 2
#
sainfo address 94.159.37.114 [500] 47 address 79.120.78.118 [500] 47 {
pfs_group modp768;
lifetime time 60 min;
encryption_algorithm des;
authentication_algorithm hmac_sha1, non_auth;
compression_algorithm deflate;
}
sainfo address 79.120.78.118 [500] 47 address 94.159.37.114 [500] 47 {
pfs_group modp768;
lifetime time 60 min;
encryption_algorithm des;
authentication_algorithm hmac_sha1, non_auth;
compression_algorithm deflate;
}
#
# Moscow, Tashkentskaya 12-20
#
sainfo address 94.159.37.114 [500] 47 address 79.120.80.66 [500] 47 {
pfs_group modp768;
lifetime time 60 min;
encryption_algorithm des;
authentication_algorithm hmac_sha1, non_auth;
compression_algorithm deflate;
}
sainfo address 79.120.80.66 [500] 47 address 94.159.37.114 [500] 47 {
pfs_group modp768;
lifetime time 60 min;
encryption_algorithm des;
authentication_algorithm hmac_sha1, non_auth;
compression_algorithm deflate;
}
#
# Moscow, Baykalskaya 50/7
#
sainfo address 94.159.37.114 [500] 47 address 213.33.223.158 [500] 47 {
pfs_group modp768;
lifetime time 60 min;
encryption_algorithm des;
authentication_algorithm hmac_sha1, non_auth;
compression_algorithm deflate;
}
sainfo address 213.33.223.158 [500] 47 address 94.159.37.114 [500] 47 {
pfs_group modp768;
lifetime time 60 min;
encryption_algorithm des;
authentication_algorithm hmac_sha1, non_auth;
compression_algorithm deflate;
}
#
# Moscow, Bratyslavskaya 15-1
#
#sainfo address 94.159.37.114 [500] 47 address 212.46.203.106 [500] 47 {
# pfs_group modp768;
# lifetime time 60 min;
# encryption_algorithm des;
# authentication_algorithm hmac_sha1, non_auth;
# compression_algorithm deflate;
#}
#sainfo address 212.46.203.106 [500] 47 address 94.159.37.114 [500] 47 {
# pfs_group modp768;
# lifetime time 60 min;
# encryption_algorithm des;
# authentication_algorithm hmac_sha1, non_auth;
# compression_algorithm deflate;
#}
>How-To-Repeat:
Get a FreeBSD 9.0, get an ipsec setup, get a bunch of gre tunnels, get a quagga (trap mentions its ospfd), and probably get a carp(4) interface (not sure if it will trap without; at least I did not test it without a carp(4)).
>Fix:
>Release-Note:
>Audit-Trail:
>Unformatted:
More information about the freebsd-bugs
mailing list