[Bug 237800] pow(3) returns inaccurate results

[bugzilla-noreply at freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=237800

--- Comment #14 from Steve Kargl <kargl at FreeBSD.org> ---
(In reply to me from comment #13)
>
> The test case I have at the moment is 0.6840442338072671 ** 2.4.
> As I understand it, it should be 0.4019777798321958 but I am
> getting 0.40197777983219574.

The first number contains 16 digits. The second contains
17 digits, so cannot make a judgement on rounding.  But,

% ./testd -a 0.6840442338072671 2.4
libm =  4.0197777983219574e-01, /* 0x3fd9ba01, 0x02864520 */
mpfr =  4.0197777983219579e-01, /* 0x3fd9ba01, 0x02864521 */
 ulp = 0.51122

MPFR is an arbitrary precision library where I have it set
to use 212 bits of precision to compute pow().  pow() is
FreeBSD's libm value with 53-bits of precision.  You have
a ULP of 0.51122.  That is about as accurate as one might
expect.

See comment #8 for a pointer to Goldberg's paper.

-- 
You are receiving this mail because:
You are the assignee for the bug.