Bad error computing 1/z
Stephen Montgomery-Smith
stephen at missouri.edu
Mon Jul 30 00:51:23 UTC 2012
As I was debugging catanh, I noticed the following oddness.
If z = cpack(x,y), where
x = 1
y = 0x1.25691d4068c910p+512, approximately 1.53672e+154
then the real part of 1/z is wrong by about 4ULP. The real part of 1/z
is approx 4.2346e-309. I know the reciprocal of this would cause an
overflow.
I tried writing my own reciprocal algorithm, and it had exactly the same
error.
Does anyone have an explanation?
I thought it might be that the number was somehow "denormalized" and it
couldn't store enough digits in the mantissa. Or maybe the error is
when mpfr converts the correct answer to double. I don't know.
More information about the freebsd-numerics
mailing list