(2nd time) tweaks to erff() threshold values

Steve Kargl sgk at troutmask.apl.washington.edu
Sun Aug 25 17:58:18 UTC 2013


On Sun, Aug 25, 2013 at 10:19:10AM -0700, Steve Kargl wrote:
> 
> Note**2, I did not record a domain and range as my routine that
> generates plots seems to give a really messed up result for
> the range [-0.08, 0.06].  However, the error estimate from
> solving the Remes matrix equation is err = 0x1.39c84809b7ed2p-38.

Found the bug in the graphing routine.

/*
 *  Domain [0.84375, 1.25], range ~[-1.954e-10,1.940e-11]:
 *  |(erf(x) - erx) - p(x)/q(x)| < 2**-36.
 */
> +pa0  =  1.35131621e-08F, /*  0x1.d04f08p-27 */
> +pa1  =  4.15107518e-01F, /*  0x1.a911f2p-2 */
> +pa2  = -1.63339108e-01F, /* -0x1.4e84bcp-3 */
> +pa3  =  1.12098485e-01F, /*  0x1.cb27c8p-4 */
> +qa1  =  6.06513679e-01F, /*  0x1.3688f6p-1 */
> +qa2  =  5.43227255e-01F, /*  0x1.1621e2p-1 */
> +qa3  =  1.74396917e-01F, /*  0x1.652a36p-3 */
> +qa4  =  5.88681065e-02F, /*  0x1.e23f5ep-5 */

Note, the range and 2**-36 estimate are from the results of
the Remes algorithm run with 1024-bits of precision.  The
above coefficients were simply rounded to single precision,
as I haven't investigated a better method for running, yet.

-- 
Steve


More information about the freebsd-numerics mailing list