Bad error computing 1/z

Stephen Montgomery-Smith stephen at missouri.edu
Mon Jul 30 00:51:23 UTC 2012


As I was debugging catanh, I noticed the following oddness.

If z = cpack(x,y), where
x = 1
y = 0x1.25691d4068c910p+512, approximately 1.53672e+154
then the real part of 1/z is wrong by about 4ULP.  The real part of 1/z 
is approx 4.2346e-309.  I know the reciprocal of this would cause an 
overflow.

I tried writing my own reciprocal algorithm, and it had exactly the same 
error.

Does anyone have an explanation?

I thought it might be that the number was somehow "denormalized" and it 
couldn't store enough digits in the mantissa.  Or maybe the error is 
when mpfr converts the correct answer to double.  I don't know.



More information about the freebsd-numerics mailing list