SHA1_Update() produces wrong results for large buffers

[Pavel Gorshkov]

On Tue, Jan 10, 2006 at 02:10:33PM +1100, Peter Jeremy wrote:
> I get this on 7-current as well.  Copying the relevant bits from libmd
> and compiling it myself, I get the same behaviour.  The fact that this
> exits virtually instantly strongly suggests that it is broken (rather
> than the shared version).  My initial guess is that an operation on
> the length is overflowing 32 bits.  Unfortunately, the asm is rather
> opaque - it was auto-generated by a perl script that doesn't seem to
> included in the repository.  (There is a sha1-586.pl in openssl but
> it generates different code).

Yes, the SHA1 implementation in libmd.a is very similar to the one
found in libcrypto.a, the former is just much older.

SHA1_Update as provided by libcrypto.a has no problems with large
buffers.  Here's a slightly modified test program (attached), now
compatible with -lcrypto:

gcc sha1test.c -o sha1test.ssl-static -lcrypto -static
gcc sha1test.c -o sha1test.md-shared  -lmd
gcc sha1test.c -o sha1test.md-static  -lmd -static

dd if=/dev/zero bs=32M count=48 of=test-1.5G
for i in ssl-static md-{shared,static}; do ./sha1test.$i test-1.5G; done
a957f01b1a92366c7b72296cb24eb84f42ed06e4
a957f01b1a92366c7b72296cb24eb84f42ed06e4
747cd7172ce7737d1735cf936c0d69ce0f733fcd


According to this page:

http://www.freebsd.org/cgi/cvsweb.cgi/src/lib/libmd/i386/

our version of `sha.S' is 6 years old, whereas `sha1-586.pl' as
found in the openssl cvs tree has underwent many changes since
then:

http://cvs.openssl.org/rlog?f=openssl/crypto/sha/asm/sha1-586.pl


> As far as I can determine, the asm code (sha1_block_x86) is designed
> to process an integral number of SHA1 blocks of input, leaving the
> remainder to be processed in the C code.  Using the debugger, the
> asm code is not looping when passed 1610612736 (1.5G) - which explains
> the rapid exit and incorrect result.

Yes, thanks for the additional info.
It looks like some parts of libmd should be either fixed/brought
in sync with the openssl cvs, or marked as deprecated.


Peter & Simon, thanks for confirming the test results.

--
Pavel Gorshkov
-------------- next part --------------
#include <sys/types.h>
#include <sys/stat.h>
#include <sys/mman.h>
#include <unistd.h>
#include <fcntl.h>
#include <stdlib.h>
#include <stdio.h>
#include <sha.h>

int main(int argc, char **argv) {
    int fd, i;
    struct stat st;
    SHA_CTX ctx;
    unsigned char *buf, digest[20];
    char hexdigest[41];

    if (argc < 2 || stat(argv[1], &st) < 0 || (fd=open(argv[1], O_RDONLY)) < 0)
	exit(1);
    
    if ((buf = mmap(NULL, st.st_size, PROT_READ, MAP_PRIVATE, fd, 0)) == MAP_FAILED)
      	exit(1);

    SHA1_Init(&ctx);
    SHA1_Update(&ctx, buf, st.st_size);
    SHA1_Final(digest, &ctx);
    for (i = 0; i < 20; ++i)
	sprintf(hexdigest + 2*i, "%02x", digest[i]);
    puts(hexdigest);

    if (st.st_size)
	munmap(buf, st.st_size);
    close(fd);
    return 0;
}