[CFR] OpenSSL ENGINE fix

[Peter Pentchev]

On Thu, Sep 02, 2004 at 07:03:00PM +0300, Peter Pentchev wrote:
> Hi,
> 
> The stunnel port had been badly broken on -CURRENT for some time, and
> today I seem to have tracked it down.  There is a problem in
> OpenSSL's ENGINE code, which seems to depend on realloc()'s initializing
> unused memory with zeroes.  Since this is certainly not true with
> malloc's 'J' option, the ENGINE code never actually runs out of RAND
> routines to look for, and dumps core on attempting to dereference a
> structure at 0xd0d0d0d0.
> 
> The following simple patch seems to fix this particular problem, but I
> think there are others lurking close beneath the surface.  When I built
> stunnel with this patch, it did not immediately segfault on
> initialization, but it did later, when it attempted to get a couple of
> random bytes in order to actually establish an SSL connection.  A quick
> examination shows that the ENGINE code had "successfully" loaded and
> initialized all built-in engines and then tried to use the last one
> loaded - in this case, the 4758cca one - which promptly invoked a
> function pointed to by the static randomNumberGenerate variable, which,
> as you may have guessed by this point, held a NULL value.  Apparently,
> something else is rotten in ENGINE initialization land, and (at least)
> the 4758cca driver cannot properly detect that it has not properly
> detected its hardware :(

Oh, and of course, here's a program that demonstrates the null pointer
problem after the eng_table.c patch is applied:

#include <openssl/lhash.h>
#include <openssl/ssl.h>
#include <openssl/err.h>
#include <openssl/crypto.h> /* for CRYPTO_* and SSLeay_version */
#include <openssl/rand.h>
#if SSLEAY_VERSION_NUMBER >= 0x00907000L
#include <openssl/engine.h>
#else
#error weird openssl version
#endif

int main(void)
{
	unsigned char buf[256];
	int ret;
	
	ENGINE_load_builtin_engines();
	ENGINE_register_all_complete();
	if (RAND_status()) {
		printf("RAND_status said ok\n");
	} else {
		printf("RAND_status kinda sorta failed\n");
	}
	ret = RAND_pseudo_bytes(buf, sizeof(buf) / 2);
	printf("RAND_pseudo_bytes() returned %d\n", ret);
	return (0);
}

G'luck,
Peter

-- 
Peter Pentchev	roam at ringlet.net    roam at cnsys.bg    roam at FreeBSD.org
PGP key:	http://people.FreeBSD.org/~roam/roam.key.asc
Key fingerprint	FDBA FD79 C26F 3C51 C95E  DF9E ED18 B68D 1619 4553
If I had finished this sentence,
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 187 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20040902/9dc56b61/attachment.bin