misc/104422: TCP-MD5 key length limits

[Bryce Riner]

>Number:         104422
>Category:       misc
>Synopsis:       TCP-MD5 key length limits
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    freebsd-bugs
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Sat Oct 14 16:30:19 GMT 2006
>Closed-Date:
>Last-Modified:
>Originator:     Bryce Riner
>Release:        6.1
>Organization:
>Environment:
FreeBSD xorp.emerson.edu 6.1-RELEASE FreeBSD 6.1-RELEASE #2: Sat Oct 14 12:00:10 EDT 2006     root at xorp.emerson.edu:/usr/src/sys/i386/compile/ROUTER  i386
>Description:
When trying to add a TCP-MD5 key with length 11 octets (88 bits) I get "The result of line 3: Invalid argument." from "setkey -f" and with IPSEC_DEBUG on "key_mature: invalid AH key length 88 (1-80 allowed)" out of dmesg.

Now the RFC says this about key length:
"It is  strongly recommended that an implementation be able to support at minimum a key composed of a string of printable ASCII of 80 bytes or less, as this is current practice."

The setkey man page has this in it's table:
"
algorithm       keylen (bits)   comment
..
tcp-md5         8 to 640        tcp: rfc2385
"

It would appear that the kernel is checking the key length in bits against min/max in octets.
>How-To-Repeat:

>Fix:
Change documentation to properly reflect kernel behavior.
Change kernel to properly check key sizes.
Use only 10 octet or 80 bit long keys.
>Release-Note:
>Audit-Trail:
>Unformatted: