misc/170519: OPIE doesn't properly do SHA-1 (otp-sha)

[Arthur Mesh]

>Number:         170519
>Category:       misc
>Synopsis:       OPIE doesn't properly do SHA-1 (otp-sha)
>Confidential:   no
>Severity:       serious
>Priority:       low
>Responsible:    freebsd-bugs
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Fri Aug 10 00:10:01 UTC 2012
>Closed-Date:
>Last-Modified:
>Originator:     arthurmesh
>Release:        FreeBSD 9.0-STABLE amd64
>Organization:
none
>Environment:
System: FreeBSD alpha 9.0-STABLE FreeBSD 9.0-STABLE #0 r235829: Wed May 23 11:03:56 PDT 2012 root at alpha:/usr/obj/usr/src/sys/GENERIC amd64


>Description:

OPIE doesn't properly do SHA1. OPIE doesn't properly implement RFC 2289, see fix
for more details.

Quote from RFC 2289  A One-Time Password System:

Appendix A:
...
   For historical reasons, and to promote interoperability with existing
   implementations, it was decided that ALL hashes incorporated into the
   OTP protocol MUST store the output of their hash function in LITTLE
   ENDIAN format BEFORE the bit folding to 64 bits occurs.  This is done
   in the implementations of MD4 and MD5 (see references [2] and [6]),
   while it must be explicitly done for the implementation of SHA1 (see
   reference [7]).

>How-To-Repeat:

# SHA1 problem:

   # On FreeBSD9
   $ echo aaaaaaaaaa | otp-sha 1 foobar
   Using the SHA-1 algorithm to compute response.
   Reminder: Don't use opiekey from telnet or dial-in sessions.
   Enter secret pass phrase: 
   KERN RUSS BETH SAUL YANG GO
   
   # On OpenBSD 5.1
   $ skey -sha1 -p aaaaaaaaaa 1 foobar   
   ROWS GIBE NOTE OAF GASH HECK

# Yet, MD5 works fine:

   # On FreeBSD9
   $ echo aaaaaaaaaa | otp-md5 1 foobar
   Using the MD5 algorithm to compute response.
   Reminder: Don't use opiekey from telnet or dial-in sessions.
   Enter secret pass phrase: 
   VETO ODIN WOO SHOD REID ROSE
   
   # On OpenBSD 5.1
   # skey -md5 -p aaaaaaaaaa 1 foobar  
   VETO ODIN WOO SHOD REID ROSE

>Fix:

Index: contrib/opie/libopie/hash.c
===================================================================
--- contrib/opie/libopie/hash.c	(revision 235829)
+++ contrib/opie/libopie/hash.c	(working copy)
@@ -17,6 +17,8 @@
 $FreeBSD$
 */
 
+#include <sys/endian.h>
+
 #include "opie_cfg.h"
 #include "opie.h"
 
@@ -32,11 +34,21 @@
   switch(algorithm) {
     case 3:
       {
+      int i;
       SHA_CTX sha;
       UINT4 digest[5];
       SHA1_Init(&sha);
       SHA1_Update(&sha, (unsigned char *)x, 8);
       SHA1_Final((unsigned char *)digest, &sha);
+
+      /*
+       * RFC2289 mandates that we convert SHA1 digest from big-endian to little
+       * see Appendix A.
+       */
+      for (i = 0; i < 5; i++) {
+          digest[i] = bswap32(digest[i]);
+      }
+
       results[0] = digest[0] ^ digest[2] ^ digest[4];
       results[1] = digest[1] ^ digest[3];
       };
Index: contrib/opie/libopie/hashlen.c
===================================================================
--- contrib/opie/libopie/hashlen.c	(revision 235829)
+++ contrib/opie/libopie/hashlen.c	(working copy)
@@ -14,6 +14,8 @@
 $FreeBSD$
 */
 
+#include <sys/endian.h>
+
 #include "opie_cfg.h"
 #include "opie.h"
 
@@ -29,11 +31,20 @@
 
   switch(algorithm) {
     case 3: {
+      int i;
       SHA_CTX sha;
       UINT4 digest[5];
       SHA1_Init(&sha);
       SHA1_Update(&sha, (unsigned char *)in, n);
       SHA1_Final((unsigned char *)digest, &sha);
+
+      /*
+       * RFC2289 mandates that we convert SHA1 digest from big-endian to little
+       * see Appendix A.
+       */
+      for (i = 0; i < 5; i++) {
+          digest[i] = bswap32(digest[i]);
+      }
       results[0] = digest[0] ^ digest[2] ^ digest[4];
       results[1] = digest[1] ^ digest[3];
       break;
>Release-Note:
>Audit-Trail:
>Unformatted: