OpenBGPd + TCP-MD5 sig fails after a few weeks

Antoine Beaupré anarcat at koumbit.org
Tue Nov 26 23:41:49 UTC 2013


[please CC me I am not on the list. also cc'ing bzeeb since he was
working on a patch for this two years ago]

Hi,

I have configured an OpenBGPd daemon to connect to another provider with a
TCP-MD5 password.

It used to work when I set it up 30 days ago, but somehow the session is
down now.

# bgpctl show
Neighbor                   AS    MsgRcvd    MsgSent  OutQ Up/Down State/PrfRcvd
Cogent                    174          0          0     0 Never   Active

I suspect this was working only because the remote server was
initializing the connexion and not us. 

What is ackward is that OpenBGPd doesn't seem to properly initialise the
socket with an MD5 signature:

17:54:59.479900 IP (tos 0xc0, ttl 1, id 14983, offset 0, flags [DF], proto TCP (6), length 60, bad cksum 0 (->c0d9)!)
    38.104.152.102.16295 > 38.104.152.101.179: Flags [S], cksum 0x096d (correct), seq 1556933933, win 65535, options [mss 1460,nop,wscale 6,sackOK,TS val 1324688 ecr 0], length 0
17:54:59.480593 IP (tos 0x0, ttl 255, id 30414, offset 0, flags [none],proto TCP (6), length 40)
    38.104.152.101.179 > 38.104.152.102.16295: Flags [R.], cksum 0xa7df (correct), seq 0, ack 1556933934, win 0, length 0

Usually, this should mention "md5valid" in the options field if it is
enabled.

This actually works with netcat:

# nc -v -S 38.104.152.101 179
nc: connect to 38.104.152.101 port 179 (tcp) failed: Connection refused

# tcpdump -M [...] -i bge0 -n -vvv port 179
tcpdump: listening on bge0, link-type EN10MB (Ethernet), capture size
65535 bytes
18:15:43.534592 IP (tos 0x0, ttl 64, id 27666, offset 0, flags [DF], proto TCP (6), length 80, bad cksum 0 (->50fa)!)
    38.104.152.102.26043 > 38.104.152.101.179: Flags [S], cksum 0xe73a (correct), seq 3803575904, win 65535, options [mss 1460,nop,wscale 6,sackOK,TS val 2568742 ecr 0,nop,nop,md5valid], length 0
18:15:43.536592 IP (tos 0x0, ttl 255, id 30526, offset 0, flags [none], proto TCP (6), length 60)
    38.104.152.101.179 > 38.104.152.102.26043: Flags [R.], cksum 0x1566 (correct), seq 0, ack 3803575905, win 0, options [md5valid,eol], length 0

Notice, however, how the other side is still reseting our connexion, so
there's something wrong there - but it could be that they simply blocked
us because of too many failed attempts.

The SAD association is set properly:

# setkey -D
38.104.152.102 38.104.152.101
        tcp mode=any spi=4096(0x00001000) reqid=0(0x00000000)
        A: tcp-md5  [...]
        seq=0x00000000 replay=0 flags=0x00000040 state=mature
        created: Nov 26 17:37:59 2013   current: Nov 26 17:57:40 2013
        diff: 1181(s)   hard: 0(s)      soft: 0(s)
        last:                           hard: 0(s)      soft: 0(s)
        current: 0(bytes)       hard: 0(bytes)  soft: 0(bytes)
        allocated: 0    hard: 0 soft: 0
        sadb_seq=1 pid=31485 refcnt=1
38.104.152.101 38.104.152.102
        tcp mode=any spi=4096(0x00001000) reqid=0(0x00000000)
        A: tcp-md5  [...]
        seq=0x00000000 replay=0 flags=0x00000040 state=mature
        created: Nov 26 17:37:59 2013   current: Nov 26 17:57:40 2013
        diff: 1181(s)   hard: 0(s)      soft: 0(s)
        last:                           hard: 0(s)      soft: 0(s)
        current: 0(bytes)       hard: 0(bytes)  soft: 0(bytes)
        allocated: 0    hard: 0 soft: 0
        sadb_seq=0 pid=31485 refcnt=1

here's our ipsec.conf:

add -n 38.104.152.101 38.104.152.102 tcp 0x1000 -A tcp-md5 "[...]";
add -n 38.104.152.102 38.104.152.101 tcp 0x1000 -A tcp-md5 "[...]";

I have tried both openbgpd-5.2.20121209 and openbgpd-5.2.20121014. I
have also tried to disable TCP verification through sysctl, no luck:

net.inet.tcp.signature_verify_input: 0

We have the following kernel configuration:

include GENERIC
ident KOUMBIT1
device          pf
device          pflog
device          pfsync
options         ALTQ
options         ALTQ_CBQ
options         ALTQ_RED
options         ALTQ_RIO
options         ALTQ_HFSC
options         ALTQ_CDNR
options         ALTQ_PRIQ
options   IPSEC        #IP security
options TCP_SIGNATURE
device    crypto
options         DEVICE_POLLING
device          carp

Our bgpd.conf doesn't specify a tcp password, otherwise we end up with
the following error:

root at rtr0:/usr/local/etc # bgpd -d
startup
rereading config
no kernel support for PF_KEY
route decision engine ready
session engine ready
RDE reconfigured
listening on 0.0.0.0
listening on ::
SE reconfigured
neighbor 38.104.152.101 (Cogent): state change None -> Idle, reason: None
neighbor 38.104.152.101 (Cogent): pfkey setup failed

Why is it that outgoing TCP connexions do not respect setkey settings?

I read a little bit of the source code, and it seems that openbgpd is
stuck in a catch-22: it can't setup the SAD associations (because they
are handled by setkey) so it doesn't set the MD5SIG option on the
socket... One horrible solution I found was to change session_connect()
as such:

-       if (peer->conf.auth.method != AUTH_NONE && sysdep.no_pfkey) {
+       if (peer->conf.auth.method != AUTH_NONE && sysdep.no_pfkey || 1) {
                log_peer_warnx(&peer->conf,
-                   "ipsec or md5sig configured but not available");
-               bgp_fsm(peer, EVNT_CON_OPENFAIL);
-               return (-1);
+                   "ipsec or md5sig configured but not available, assuming set externally");
+               /* bgp_fsm(peer, EVNT_CON_OPENFAIL); */
+               /* return (-1); */
+               if (setsockopt(peer->fd, IPPROTO_TCP, TCP_MD5SIG,
+                    &opt, sizeof(opt)) == -1) {
+                        log_peer_warn(&peer->conf, "setsockopt md5sig");
+                        bgp_fsm(peer, EVNT_CON_OPENFAIL);
+                        return (-1);
+                }
+
        }

It's pretty nasty, but with this at least the connexion gets initialized
going out with the right socket options.

A.

PS: this is a problem similar to what was reported here:

http://lists.freebsd.org/pipermail/freebsd-net/2012-January/030921.html

-- 
Il faut tout un village pour élever un enfant.
                        - Proverbe africain
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: <http://lists.freebsd.org/pipermail/freebsd-net/attachments/20131126/9586239c/attachment.sig>


More information about the freebsd-net mailing list