OpenBGPd + TCP-MD5 sig fails after a few weeks
Ermal Luçi
eri at freebsd.org
Wed Nov 27 10:58:13 UTC 2013
On Wed, Nov 27, 2013 at 12:41 AM, Antoine Beaupré <anarcat at koumbit.org>wrote:
> [please CC me I am not on the list. also cc'ing bzeeb since he was
> working on a patch for this two years ago]
>
> Hi,
>
> I have configured an OpenBGPd daemon to connect to another provider with a
> TCP-MD5 password.
>
> It used to work when I set it up 30 days ago, but somehow the session is
> down now.
>
> # bgpctl show
> Neighbor AS MsgRcvd MsgSent OutQ Up/Down
> State/PrfRcvd
> Cogent 174 0 0 0 Never Active
>
> I suspect this was working only because the remote server was
> initializing the connexion and not us.
>
> What is ackward is that OpenBGPd doesn't seem to properly initialise the
> socket with an MD5 signature:
>
> 17:54:59.479900 IP (tos 0xc0, ttl 1, id 14983, offset 0, flags [DF], proto
> TCP (6), length 60, bad cksum 0 (->c0d9)!)
> 38.104.152.102.16295 > 38.104.152.101.179: Flags [S], cksum 0x096d
> (correct), seq 1556933933, win 65535, options [mss 1460,nop,wscale
> 6,sackOK,TS val 1324688 ecr 0], length 0
> 17:54:59.480593 IP (tos 0x0, ttl 255, id 30414, offset 0, flags
> [none],proto TCP (6), length 40)
> 38.104.152.101.179 > 38.104.152.102.16295: Flags [R.], cksum 0xa7df
> (correct), seq 0, ack 1556933934, win 0, length 0
>
> Usually, this should mention "md5valid" in the options field if it is
> enabled.
>
> This actually works with netcat:
>
> # nc -v -S 38.104.152.101 179
> nc: connect to 38.104.152.101 port 179 (tcp) failed: Connection refused
>
> # tcpdump -M [...] -i bge0 -n -vvv port 179
> tcpdump: listening on bge0, link-type EN10MB (Ethernet), capture size
> 65535 bytes
> 18:15:43.534592 IP (tos 0x0, ttl 64, id 27666, offset 0, flags [DF], proto
> TCP (6), length 80, bad cksum 0 (->50fa)!)
> 38.104.152.102.26043 > 38.104.152.101.179: Flags [S], cksum 0xe73a
> (correct), seq 3803575904, win 65535, options [mss 1460,nop,wscale
> 6,sackOK,TS val 2568742 ecr 0,nop,nop,md5valid], length 0
> 18:15:43.536592 IP (tos 0x0, ttl 255, id 30526, offset 0, flags [none],
> proto TCP (6), length 60)
> 38.104.152.101.179 > 38.104.152.102.26043: Flags [R.], cksum 0x1566
> (correct), seq 0, ack 3803575905, win 0, options [md5valid,eol], length 0
>
> Notice, however, how the other side is still reseting our connexion, so
> there's something wrong there - but it could be that they simply blocked
> us because of too many failed attempts.
>
> The SAD association is set properly:
>
> # setkey -D
> 38.104.152.102 38.104.152.101
> tcp mode=any spi=4096(0x00001000) reqid=0(0x00000000)
> A: tcp-md5 [...]
> seq=0x00000000 replay=0 flags=0x00000040 state=mature
> created: Nov 26 17:37:59 2013 current: Nov 26 17:57:40 2013
> diff: 1181(s) hard: 0(s) soft: 0(s)
> last: hard: 0(s) soft: 0(s)
> current: 0(bytes) hard: 0(bytes) soft: 0(bytes)
> allocated: 0 hard: 0 soft: 0
> sadb_seq=1 pid=31485 refcnt=1
> 38.104.152.101 38.104.152.102
> tcp mode=any spi=4096(0x00001000) reqid=0(0x00000000)
> A: tcp-md5 [...]
> seq=0x00000000 replay=0 flags=0x00000040 state=mature
> created: Nov 26 17:37:59 2013 current: Nov 26 17:57:40 2013
> diff: 1181(s) hard: 0(s) soft: 0(s)
> last: hard: 0(s) soft: 0(s)
> current: 0(bytes) hard: 0(bytes) soft: 0(bytes)
> allocated: 0 hard: 0 soft: 0
> sadb_seq=0 pid=31485 refcnt=1
>
> here's our ipsec.conf:
>
> add -n 38.104.152.101 38.104.152.102 tcp 0x1000 -A tcp-md5 "[...]";
> add -n 38.104.152.102 38.104.152.101 tcp 0x1000 -A tcp-md5 "[...]";
>
> I have tried both openbgpd-5.2.20121209 and openbgpd-5.2.20121014. I
> have also tried to disable TCP verification through sysctl, no luck:
>
> net.inet.tcp.signature_verify_input: 0
>
> We have the following kernel configuration:
>
> include GENERIC
> ident KOUMBIT1
> device pf
> device pflog
> device pfsync
> options ALTQ
> options ALTQ_CBQ
> options ALTQ_RED
> options ALTQ_RIO
> options ALTQ_HFSC
> options ALTQ_CDNR
> options ALTQ_PRIQ
> options IPSEC #IP security
> options TCP_SIGNATURE
> device crypto
> options DEVICE_POLLING
> device carp
>
> Our bgpd.conf doesn't specify a tcp password, otherwise we end up with
> the following error:
>
> root at rtr0:/usr/local/etc # bgpd -d
> startup
> rereading config
> no kernel support for PF_KEY
> route decision engine ready
> session engine ready
> RDE reconfigured
> listening on 0.0.0.0
> listening on ::
> SE reconfigured
> neighbor 38.104.152.101 (Cogent): state change None -> Idle, reason: None
> neighbor 38.104.152.101 (Cogent): pfkey setup failed
>
> Why is it that outgoing TCP connexions do not respect setkey settings?
>
> I read a little bit of the source code, and it seems that openbgpd is
> stuck in a catch-22: it can't setup the SAD associations (because they
> are handled by setkey) so it doesn't set the MD5SIG option on the
> socket... One horrible solution I found was to change session_connect()
> as such:
>
> - if (peer->conf.auth.method != AUTH_NONE && sysdep.no_pfkey) {
> + if (peer->conf.auth.method != AUTH_NONE && sysdep.no_pfkey || 1) {
> log_peer_warnx(&peer->conf,
> - "ipsec or md5sig configured but not available");
> - bgp_fsm(peer, EVNT_CON_OPENFAIL);
> - return (-1);
> + "ipsec or md5sig configured but not available,
> assuming set externally");
> + /* bgp_fsm(peer, EVNT_CON_OPENFAIL); */
> + /* return (-1); */
> + if (setsockopt(peer->fd, IPPROTO_TCP, TCP_MD5SIG,
> + &opt, sizeof(opt)) == -1) {
> + log_peer_warn(&peer->conf, "setsockopt md5sig");
> + bgp_fsm(peer, EVNT_CON_OPENFAIL);
> + return (-1);
> + }
> +
> }
>
> It's pretty nasty, but with this at least the connexion gets initialized
> going out with the right socket options.
>
>
You can use the port here
https://github.com/pfsense/pfsense-tools/tree/master/pfPorts/openbgpd
It has integration with pfkey sockets of FreeBSD in the daemon itself and
you have to specify only th espd policy through setkey.
It works for pfSense.
> A.
>
> PS: this is a problem similar to what was reported here:
>
> http://lists.freebsd.org/pipermail/freebsd-net/2012-January/030921.html
>
> --
> Il faut tout un village pour élever un enfant.
> - Proverbe africain
>
--
Ermal
More information about the freebsd-net
mailing list