(2nd time) tweaks to erff() threshold values
Steve Kargl
sgk at troutmask.apl.washington.edu
Sun Aug 25 17:58:18 UTC 2013
On Sun, Aug 25, 2013 at 10:19:10AM -0700, Steve Kargl wrote:
>
> Note**2, I did not record a domain and range as my routine that
> generates plots seems to give a really messed up result for
> the range [-0.08, 0.06]. However, the error estimate from
> solving the Remes matrix equation is err = 0x1.39c84809b7ed2p-38.
Found the bug in the graphing routine.
/*
* Domain [0.84375, 1.25], range ~[-1.954e-10,1.940e-11]:
* |(erf(x) - erx) - p(x)/q(x)| < 2**-36.
*/
> +pa0 = 1.35131621e-08F, /* 0x1.d04f08p-27 */
> +pa1 = 4.15107518e-01F, /* 0x1.a911f2p-2 */
> +pa2 = -1.63339108e-01F, /* -0x1.4e84bcp-3 */
> +pa3 = 1.12098485e-01F, /* 0x1.cb27c8p-4 */
> +qa1 = 6.06513679e-01F, /* 0x1.3688f6p-1 */
> +qa2 = 5.43227255e-01F, /* 0x1.1621e2p-1 */
> +qa3 = 1.74396917e-01F, /* 0x1.652a36p-3 */
> +qa4 = 5.88681065e-02F, /* 0x1.e23f5ep-5 */
Note, the range and 2**-36 estimate are from the results of
the Remes algorithm run with 1024-bits of precision. The
above coefficients were simply rounded to single precision,
as I haven't investigated a better method for running, yet.
--
Steve
More information about the freebsd-numerics
mailing list