(2nd time) tweaks to erff() threshold values

Mon Aug 26 17:43:14 UTC 2013

On Tue, Aug 27, 2013 at 01:23:19AM +1000, Bruce Evans wrote:
> On Sun, 25 Aug 2013, Steve Kargl wrote:
>
>> These values where for the interval [0,0.84375].
>>>
>>>          | usec/call | Max ULP | X Max ULP
>>> ---------+-----------+---------+---------------------------------
>>> new erfc |  0.02757  | 0.65947 | 8.19824636e-01, 0x1.a3c00ep-1
>>> old erfc |  0.03348  | 0.68218 | 8.43257010e-01, 0x1.afbf62p-1
>>> ---------+-----------+---------+---------------------------------
>>> new erf  |  0.02302  | 0.62437 | 1.04175471e-38, 0x1.c5bf88p-127
>>> old erf  |  0.03028  | 0.62437 | 1.04175471e-38, 0x1.c5bf88p-127
>
> Be careful testing on i386.  Its extra precision gives smaller errors.
> These look like the i386 errors.  Old erff has a maximum error of more
> than 0.9 ulps on all args.

Have I ever mentioned how much I dislike the i386? :-)
New test results for both i386 and amd64 follow my .sig.
New diff that merges all previous diff follows the test
results.

> > Note that s_erf.c claims that in this interval, a taylor series about
> > erf(1) is used and suggests that the constant erx =  8.4506291151e-01
> > is erf(1).  That's bogus as erf(1) = 8.42700779e-01F (in float).
>
> Well, not quite.  It says:
>
> %  *	   Remark: here we use the taylor series expansion at x=1.
> %  *		erf(1+s) = erf(1) + s*Poly(s)
> %  *			 = 0.845.. + P1(s)/Q1(s)
> %  *	   That is, we use rational approximation to approximate
> %  *			erf(1+s) - (c = (single)0.84506291151)
> %  *	   Note that |P1/Q1|< 0.078 for x in [0.84375,1.25]
> %  *	   where
> %  *		P1(s) = degree 6 poly in s
> %  *		Q1(s) = degree 6 poly in s
>
> It puts some of erf(1) in 0.845 and some in P1(s)/Q1(s).

I see.  I (obviously) not did glean this from the comment.
The (c = (single)0.84506291151) isn't the leading 24-bits
of erf(1).  It's the leading 24-bits of a constant near
erf(1).  I can regenerate polynomials with a hi+lo split.

> > Index: src/s_erff.c
> > ...
> > -pa0  = -2.3621185683e-03, /* 0xbb1acdc6 */
> > ...
> > -qa1  =  1.0642088205e-01, /* 0x3dd9f331 */
> > ...
> > +pa0  =  1.35131621e-08F, /*  0x1.d04f08p-27 */
> > ...
> > +qa1  =  6.06513679e-01F, /*  0x1.3688f6p-1 */
>
> (qa0 is spelled 'one' in both versions.)

Yes 'qa0' is spelled as 'one'.  I may remove this,
and spell 'qa0' as '1'.

> You still have some of erf(1) in pa0/qa0, but not as much as before.  The
> decomposition should be something like hi+lo -- write erf(1) = hi+lo and
> put hi in erx and lo in pa0/qa0 = pa0.  Is that exactly what you do?  I
> wonder if there is a technical reason for putting more in pa0/qa0.

I did not do any splitting.  I set erx = erf(1) in float precision.
pa0 ~ 1e-8f is on the order of epsilon, and the results of solving
a matrix equation.  Perhaps, hi+lo was used to ensure the stability
of the matrix solver.

> maybe the whole expression should be written as hi+P/Q+lo, where P(0) = 0.
> Sometimes rational approximations should be done by splitting out even
> more terms from the P/Q part and adding the other terms more carefully.
> The parts that are split out are chosen to make them easier to add up
> carefully, while keeping the parts that are not split out small so that
> they don't need to be evaluated carefully.  See k_tan.c for an example.

I left my copy of Hildebrand at work over the weekend, so I haven't
reviewed the pitfalls of rational approximations, yet.

> The comment does say that 'c' is rounded to single.  That is good for
> a hi+lo decomposition, and it may be necessary for technical reasons
> to keep some of 'c's lower bits zero.  Rounding it to single doesn't
> give this when the precision is already single.  Old erff didn't round
> it further either.

I wasn't aware of old BSD libm code.  After working through the
code in s_erf[f].c, it seemed to me that a better algorithm may
exist, but I haven't tried to look for one.  As you may guess,
I'm gearing up to translate s_erf.c into a s_erfl.c.

--
Steve

erfc  (Look at the old and new ULP for 3rd and 4th interval.  This is due
to the change in mask from 0xfffff000 to 0xffffe000.)

i386
-----+
Code |    Interval    | us/call | Max ULP | X Max ULP
-----+----------------+---------+---------+-------------------------------
Old  | [0, 0.84375]   | 0.03435 | 0.68218 | 8.43257010e-01, 0x1.afbf62p-1
Old  | [0.84375,1.25] | 0.03486 | 0.95158 | 1.24999416e+00, 0x1.3fff9ep+0
Old  | [1.25,2.857143]| 0.14772 | 5.81600 | 1.88727951e+00, 0x1.e324c0p+0
Old  | [2.857143,12]  | 0.27741 | 63.6824 | 7.99420977e+00, 0x1.ffa122p+2
-----+----------------+---------+---------+-------------------------------
New  | [0, 0.84375]   | 0.02809 | 0.65947 | 8.19824636e-01, 0x1.a3c00ep-1
New  | [0.84375,1.25] | 0.02780 | 0.76497 | 1.24971473e+00, 0x1.3fed4ep+0
New  | [1.25,2.857143]| 0.13546 | 2.10176 | 1.88294506e+00, 0x1.e208b0p+0
New  | [2.857143,12]  | 0.18424 | 1.94002 | 4.00527334e+00, 0x1.005666p+2
-----+----------------+---------+---------+-------------------------------

Amd64
-----+
Code |    Interval    | us/call | Max ULP | X Max ULP
-----+----------------+---------+---------+-------------------------------
Old  | [0, 0.84375]   | 0.02847 | 2.77642 | 8.43427539e-01, 0x1.afd5bcp-1
Old  | [0.84375,1.25] | 0.02755 | 2.35308 | 1.24922514e+00, 0x1.3fcd38p+0
Old  | [1.25,2.857143]| 0.10872 | 6.27739 | 1.88167512e+00, 0x1.e1b576p+0
Old  | [2.857143,12]  | 0.14263 | 63.8095 | 7.99418974e+00, 0x1.ffa0cep+2
-----+----------------+---------+---------+-------------------------------
New  | [0, 0.84375]   | 0.02440 | 2.77756 | 8.33204687e-01, 0x1.aa99cep-1
New  | [0.84375,1.25] | 0.02337 | 2.31164 | 1.24426317e+00, 0x1.3e8808p+0
New  | [1.25,2.857143]| 0.10051 | 3.16774 | 1.34512150e+00, 0x1.5859e2p+0
New  | [2.857143,12]  | 0.10205 | 2.90042 | 4.08927298e+00, 0x1.05b6a6p+2
-----+----------------+---------+---------+-------------------------------

erf (Not much change in ULP, but increase in speed).

i386
-----+
Code |    Interval    | usec/call | Max ULP | X Max ULP
-----+----------------+-----------+---------+--------------------------------
Old  | [0, 0.84375]   | 0.03074   | 0.62437 | 1.04175471e-38, 0x1.c5bf88p-127
Old  | [0.84375,1.25] | 0.03405   | 0.55617 | 1.24996567e+00, 0x1.3ffdc0p+0
Old  | [1.25,2.857143]| 0.14923   | 0.61903 | 1.25788522e+00, 0x1.4204c4p+0
Old  | [2.857143,12]  | 0.05959   | 0.50002 | 2.86163688e+00, 0x1.6e4a1ep+1
-----+----------------+-----------+---------+--------------------------------
New  | [0, 0.84375]   | 0.02347   | 0.62437 | 1.04175471e-38, 0x1.c5bf88p-127
New  | [0.84375,1.25] | 0.02774   | 0.54213 | 8.52452755e-01, 0x1.b474b0p-1
New  | [1.25,2.857143]| 0.14229   | 0.59585 | 1.26142204e+00, 0x1.42ec8ep+0
New  | [2.857143,12]  | 0.02932   | 0.50002 | 2.86393452e+00, 0x1.6e9568p+1
-----+----------------+-----------+---------+--------------------------------

Amd64
-----+
Code |    Interval    | us/call | Max ULP | X Max ULP
-----+----------------+---------+---------+--------------------------------
Old  | [0, 0.84375]   | 0.02699 | 0.96792 | 8.36685717e-01, 0x1.ac6212p-1
Old  | [0.84375,1.25] | 0.02912 | 0.76611 | 1.23045731e+00, 0x1.3aff4p+0
Old  | [1.25,2.857143]| 0.10780 | 0.74929 | 1.25055075e+00, 0x1.402418p+0
Old  | [2.857143,12]  | 0.04671 | 0.50008 | 2.89601469e+00, 0x1.72b09cp+1
-----+----------------+---------+---------+--------------------------------
New  | [0, 0.84375]   | 0.02241 | 0.94208 | 8.36147904e-01, 0x1.ac1b94p-1
New  | [0.84375,1.25] | 0.02486 | 0.76446 | 1.18880725e+00, 0x1.3055acp+0
New  | [1.25,2.857143]| 0.10124 | 0.72032 | 1.28225052e+00, 0x1.484192p+0
New  | [2.857143,12]  | 0.02954 | 0.50003 | 2.89976478e+00, 0x1.732b7ep+1
-----+----------------+---------+---------+--------------------------------

Index: src/s_erff.c
===================================================================
--- src/s_erff.c	(revision 1361)
+++ src/s_erff.c	(working copy)
@@ -24,75 +24,61 @@ tiny	    = 1e-30,
half=  5.0000000000e-01, /* 0x3F000000 */
one =  1.0000000000e+00, /* 0x3F800000 */
two =  2.0000000000e+00, /* 0x40000000 */
-	/* c = (subfloat)0.84506291151 */
-erx =  8.4506291151e-01, /* 0x3f58560b */
+
+erx =  8.42700779e-01F, /* 0x1.af767ap-1 */
+
/*
* Coefficients for approximation to  erf on [0,0.84375]
*/
efx =  1.2837916613e-01, /* 0x3e0375d4 */
efx8=  1.0270333290e+00, /* 0x3f8375d4 */
-pp0  =  1.2837916613e-01, /* 0x3e0375d4 */
-pp1  = -3.2504209876e-01, /* 0xbea66beb */
-pp2  = -2.8481749818e-02, /* 0xbce9528f */
-pp3  = -5.7702702470e-03, /* 0xbbbd1489 */
-pp4  = -2.3763017452e-05, /* 0xb7c756b1 */
-qq1  =  3.9791721106e-01, /* 0x3ecbbbce */
-qq2  =  6.5022252500e-02, /* 0x3d852a63 */
-qq3  =  5.0813062117e-03, /* 0x3ba68116 */
-qq4  =  1.3249473704e-04, /* 0x390aee49 */
-qq5  = -3.9602282413e-06, /* 0xb684e21a */
/*
- * Coefficients for approximation to  erf  in [0.84375,1.25]
+ *  Domain [0, 0.84375], range ~[-5.4446e-10,5.5197e-10]:
+ *  |(erf(x) - x)/x - p(x)/q(x)| < 2**-31.
+ */
+pp0  =  1.28379166e-01F,	/*  0x1.06eba8p-3 */
+pp1  = -3.36030394e-01F,	/* -0x1.58185ap-2 */
+pp2  = -1.86260219e-03F,	/* -0x1.e8451ep-10 */
+qq1  =  3.12324286e-01F,	/*  0x1.3fd1f0p-2 */
+qq2  =  2.16070302e-02F,	/*  0x1.620274p-6 */
+qq3  = -1.98859419e-03F, 	/* -0x1.04a626p-9 */
+/*
+ *  Domain [0.84375, 1.25], range ~[-1.954e-10,1.940e-11]:
+ *  |(erf(x) - erx) - p(x)/q(x)| < 2**-36.
*/
-pa0  = -2.3621185683e-03, /* 0xbb1acdc6 */
-pa1  =  4.1485610604e-01, /* 0x3ed46805 */
-pa2  = -3.7220788002e-01, /* 0xbebe9208 */
-pa3  =  3.1834661961e-01, /* 0x3ea2fe54 */
-pa4  = -1.1089469492e-01, /* 0xbde31cc2 */
-pa5  =  3.5478305072e-02, /* 0x3d1151b3 */
-pa6  = -2.1663755178e-03, /* 0xbb0df9c0 */
-qa1  =  1.0642088205e-01, /* 0x3dd9f331 */
-qa2  =  5.4039794207e-01, /* 0x3f0a5785 */
-qa3  =  7.1828655899e-02, /* 0x3d931ae7 */
-qa4  =  1.2617121637e-01, /* 0x3e013307 */
-qa5  =  1.3637083583e-02, /* 0x3c5f6e13 */
-qa6  =  1.1984500103e-02, /* 0x3c445aa3 */
+pa0  =  1.35131621e-08F, /*  0x1.d04f08p-27 */
+pa1  =  4.15107518e-01F, /*  0x1.a911f2p-2 */
+pa2  = -1.63339108e-01F, /* -0x1.4e84bcp-3 */
+pa3  =  1.12098485e-01F, /*  0x1.cb27c8p-4 */
+qa1  =  6.06513679e-01F, /*  0x1.3688f6p-1 */
+qa2  =  5.43227255e-01F, /*  0x1.1621e2p-1 */
+qa3  =  1.74396917e-01F, /*  0x1.652a36p-3 */
+qa4  =  5.88681065e-02F, /*  0x1.e23f5ep-5 */
/*
- * Coefficients for approximation to  erfc in [1.25,1/0.35]
+ *  Domain [1.25,1/0.35], range [-7.043e-10,7.457e-10]:
+ *  |log(x*erfc(x)) + x**2 + 0.5625 - r(x)/s(x)| < 2**-30
*/
-ra0  = -9.8649440333e-03, /* 0xbc21a093 */
-ra1  = -6.9385856390e-01, /* 0xbf31a0b7 */
-ra2  = -1.0558626175e+01, /* 0xc128f022 */
-ra3  = -6.2375331879e+01, /* 0xc2798057 */
-ra4  = -1.6239666748e+02, /* 0xc322658c */
-ra5  = -1.8460508728e+02, /* 0xc3389ae7 */
-ra6  = -8.1287437439e+01, /* 0xc2a2932b */
-ra7  = -9.8143291473e+00, /* 0xc11d077e */
-sa1  =  1.9651271820e+01, /* 0x419d35ce */
-sa2  =  1.3765776062e+02, /* 0x4309a863 */
-sa3  =  4.3456588745e+02, /* 0x43d9486f */
-sa4  =  6.4538726807e+02, /* 0x442158c9 */
-sa5  =  4.2900814819e+02, /* 0x43d6810b */
-sa6  =  1.0863500214e+02, /* 0x42d9451f */
-sa7  =  6.5702495575e+00, /* 0x40d23f7c */
-sa8  = -6.0424413532e-02, /* 0xbd777f97 */
+ra0  = -9.87132732e-03F, /* -0x1.4376b2p-7 */
+ra1  = -5.53605914e-01F, /* -0x1.1b723cp-1 */
+ra2  = -2.17589188e+00F, /* -0x1.1683ap+1 */
+ra3  = -1.43268085e+00F, /* -0x1.6ec42cp+0 */
+sa1  =  5.45995426e+00F, /*  0x1.5d6fe4p+2 */
+sa2  =  6.69798088e+00F, /*  0x1.acabb8p+2 */
+sa3  =  1.43113089e+00F, /*  0x1.6e5e98p+0 */
+sa4  = -5.77397496e-02F, /* -0x1.d90108p-5 */
/*
- * Coefficients for approximation to  erfc in [1/.35,28]
+ *  Domain [1.25,1/0.35], range [-3.753e-12,3.875e-12]:
+ *  |log(x*erfc(x)) + x**2 + 0.5625 - r(x)/s(x)| < 2**-43
*/
-rb0  = -9.8649431020e-03, /* 0xbc21a092 */
-rb1  = -7.9928326607e-01, /* 0xbf4c9dd4 */
-rb2  = -1.7757955551e+01, /* 0xc18e104b */
-rb3  = -1.6063638306e+02, /* 0xc320a2ea */
-rb4  = -6.3756646729e+02, /* 0xc41f6441 */
-rb5  = -1.0250950928e+03, /* 0xc480230b */
-rb6  = -4.8351919556e+02, /* 0xc3f1c275 */
-sb1  =  3.0338060379e+01, /* 0x41f2b459 */
-sb2  =  3.2579251099e+02, /* 0x43a2e571 */
-sb3  =  1.5367296143e+03, /* 0x44c01759 */
-sb4  =  3.1998581543e+03, /* 0x4547fdbb */
-sb5  =  2.5530502930e+03, /* 0x451f90ce */
-sb6  =  4.7452853394e+02, /* 0x43ed43a7 */
-sb7  = -2.2440952301e+01; /* 0xc1b38712 */
+rb0  = -9.86494310e-03F, /* -0x1.434124p-7 */
+rb1  = -6.24557793e-01F, /* -0x1.3fc60ap-1 */
+rb2  = -6.12944698e+00F, /* -0x1.8848dcp+2 */
+rb3  = -1.64760838e+01F, /* -0x1.079e0ap+4 */
+rb4  = -9.36094189e+00F, /* -0x1.2b8cd6p+3 */
+sb1  =  1.26263084e+01F, /*  0x1.940ab8p+3 */
+sb2  =  4.47332840e+01F, /*  0x1.65ddc4p+5 */
+sb3  =  4.65590134e+01F, /*  0x1.7478dcp+5 */
+sb4  =  8.74471664e+00F; /*  0x1.17d4b8p+3 */

float
erff(float x)
@@ -107,43 +93,38 @@ erff(float x)
}

if(ix < 0x3f580000) {		/* |x|<0.84375 */
-	    if(ix < 0x31800000) { 	/* |x|<2**-28 */
-	        if (ix < 0x04000000)
-		    /*avoid underflow */
-		    return (float)0.125*((float)8.0*x+efx8*x);
+	    if(ix < 0x38800000) { 	/* |x|<2**-14 */
+	        if (ix < 0x04000000)	/* |x|<0x1p-119 */
+		    return (8*x+efx8*x)/8;	/* avoid spurious underflow */
return x + efx*x;
}
z = x*x;
-	    r = pp0+z*(pp1+z*(pp2+z*(pp3+z*pp4)));
-	    s = one+z*(qq1+z*(qq2+z*(qq3+z*(qq4+z*qq5))));
+	    r = pp0+z*(pp1+z*pp2);
+	    s = one+z*(qq1+z*(qq2+z*qq3));
y = r/s;
return x + x*y;
}
if(ix < 0x3fa00000) {		/* 0.84375 <= |x| < 1.25 */
s = fabsf(x)-one;
-	    P = pa0+s*(pa1+s*(pa2+s*(pa3+s*(pa4+s*(pa5+s*pa6)))));
-	    Q = one+s*(qa1+s*(qa2+s*(qa3+s*(qa4+s*(qa5+s*qa6)))));
+	    P = pa0+s*(pa1+s*(pa2+s*pa3));
+	    Q = one+s*(qa1+s*(qa2+s*(qa3+s*qa4)));
if(hx>=0) return erx + P/Q; else return -erx - P/Q;
}
-	if (ix >= 0x40c00000) {		/* inf>|x|>=6 */
+	if (ix >= 0x40800000) {		/* inf>|x|>=4 */
if(hx>=0) return one-tiny; else return tiny-one;
}
x = fabsf(x);
s = one/(x*x);
if(ix< 0x4036DB6E) {	/* |x| < 1/0.35 */
-	    R=ra0+s*(ra1+s*(ra2+s*(ra3+s*(ra4+s*(
-				ra5+s*(ra6+s*ra7))))));
-	    S=one+s*(sa1+s*(sa2+s*(sa3+s*(sa4+s*(
-				sa5+s*(sa6+s*(sa7+s*sa8)))))));
+	    R=ra0+s*(ra1+s*(ra2+s*ra3));
+	    S=one+s*(sa1+s*(sa2+s*(sa3+s*sa4)));
} else {	/* |x| >= 1/0.35 */
-	    R=rb0+s*(rb1+s*(rb2+s*(rb3+s*(rb4+s*(
-				rb5+s*rb6)))));
-	    S=one+s*(sb1+s*(sb2+s*(sb3+s*(sb4+s*(
-				sb5+s*(sb6+s*sb7))))));
+	    R=rb0+s*(rb1+s*(rb2+s*(rb3+s*rb4)));
+	    S=one+s*(sb1+s*(sb2+s*(sb3+s*sb4)));
}
GET_FLOAT_WORD(ix,x);
-	SET_FLOAT_WORD(z,ix&0xfffff000);
-	r  =  __ieee754_expf(-z*z-(float)0.5625)*__ieee754_expf((z-x)*(z+x)+R/S);
+	SET_FLOAT_WORD(z,ix&0xffffe000);
+	r  = expf(-z*z-0.5625F)*expf((z-x)*(z+x)+R/S);
if(hx>=0) return one-r/x; else return  r/x-one;
}

@@ -160,11 +141,11 @@ erfcf(float x)
}

if(ix < 0x3f580000) {		/* |x|<0.84375 */
-	    if(ix < 0x23800000)  	/* |x|<2**-56 */
+	    if(ix < 0x33800000)  	/* |x|<2**-24 */
return one-x;
z = x*x;
-	    r = pp0+z*(pp1+z*(pp2+z*(pp3+z*pp4)));
-	    s = one+z*(qq1+z*(qq2+z*(qq3+z*(qq4+z*qq5))));
+	    r = pp0+z*(pp1+z*pp2);
+	    s = one+z*(qq1+z*(qq2+z*qq3));
y = r/s;
if(hx < 0x3e800000) {  	/* x<1/4 */
return one-(x+x*y);
@@ -176,33 +157,28 @@ erfcf(float x)
}
if(ix < 0x3fa00000) {		/* 0.84375 <= |x| < 1.25 */
s = fabsf(x)-one;
-	    P = pa0+s*(pa1+s*(pa2+s*(pa3+s*(pa4+s*(pa5+s*pa6)))));
-	    Q = one+s*(qa1+s*(qa2+s*(qa3+s*(qa4+s*(qa5+s*qa6)))));
+	    P = pa0+s*(pa1+s*(pa2+s*pa3));
+	    Q = one+s*(qa1+s*(qa2+s*(qa3+s*qa4)));
if(hx>=0) {
z  = one-erx; return z - P/Q;
} else {
z = erx+P/Q; return one+z;
}
}
-	if (ix < 0x41e00000) {		/* |x|<28 */
+	if (ix < 0x41200000) {		/* |x|<10 */
x = fabsf(x);
s = one/(x*x);
if(ix< 0x4036DB6D) {	/* |x| < 1/.35 ~ 2.857143*/
-	        R=ra0+s*(ra1+s*(ra2+s*(ra3+s*(ra4+s*(
-				ra5+s*(ra6+s*ra7))))));
-	        S=one+s*(sa1+s*(sa2+s*(sa3+s*(sa4+s*(
-				sa5+s*(sa6+s*(sa7+s*sa8)))))));
+	        R=ra0+s*(ra1+s*(ra2+s*ra3));
+	        S=one+s*(sa1+s*(sa2+s*(sa3+s*sa4)));
} else {			/* |x| >= 1/.35 ~ 2.857143 */
-		if(hx<0&&ix>=0x40c00000) return two-tiny;/* x < -6 */
-	        R=rb0+s*(rb1+s*(rb2+s*(rb3+s*(rb4+s*(
-				rb5+s*rb6)))));
-	        S=one+s*(sb1+s*(sb2+s*(sb3+s*(sb4+s*(
-				sb5+s*(sb6+s*sb7))))));
+		if(hx<0&&ix>=0x40a00000) return two-tiny;/* x < -5 */
+	        R=rb0+s*(rb1+s*(rb2+s*(rb3+s*rb4)));
+		S=one+s*(sb1+s*(sb2+s*(sb3+s*sb4)));
}
GET_FLOAT_WORD(ix,x);
-	    SET_FLOAT_WORD(z,ix&0xfffff000);
-	    r  =  __ieee754_expf(-z*z-(float)0.5625)*
-			__ieee754_expf((z-x)*(z+x)+R/S);
+	    SET_FLOAT_WORD(z,ix&0xffffe000);
+	    r  = expf(-z*z-0.5625F)*expf((z-x)*(z+x)+R/S);
if(hx>0) return r/x; else return two-r/x;
} else {
if(hx>0) return tiny*tiny; else return two-tiny;