svn commit: r329259 - head/lib/msun/src

[Eitan Adler]

Author: eadler
Date: Wed Feb 14 07:59:30 2018
New Revision: 329259
URL: https://svnweb.freebsd.org/changeset/base/329259

Log:
  msun: signed overflow in atan2
  
  As a component of atan2(y, x), the case of x == 1.0 is farmed out to
  atan(y). The current implementation of this comparison is vulnerable
  to signed integer underflow (that is, undefined behavior), and it's
  performed in a somewhat more complicated way than it need be. Change
  it to not be quite so cute, rather directly comparing the high/low
  bits of x to the specific IEEE-754 bit pattern that encodes 1.0.
  
  Note that while there are three different e_atan* files in the
  relevant directory, only this one needs fixing. e_atan2f.c already
  compares against the full bit pattern encoding 1.0f, while
  e_atan2l.cuses bitwise-ands/ors/nots and so doesn't require a change.
  
  Closes #130
  
  Submitted by:	Jeff Walden (@jswalden github PR #130)
  Reviewed by:	bde
  MFC After:	1 month

Modified:
  head/lib/msun/src/e_atan2.c

Modified: head/lib/msun/src/e_atan2.c
==============================================================================
--- head/lib/msun/src/e_atan2.c	Wed Feb 14 02:51:28 2018	(r329258)
+++ head/lib/msun/src/e_atan2.c	Wed Feb 14 07:59:30 2018	(r329259)
@@ -71,7 +71,7 @@ __ieee754_atan2(double y, double x)
 	if(((ix|((lx|-lx)>>31))>0x7ff00000)||
 	   ((iy|((ly|-ly)>>31))>0x7ff00000))	/* x or y is NaN */
 	   return x+y;
-	if((hx-0x3ff00000|lx)==0) return atan(y);   /* x=1.0 */
+	if(hx==0x3ff00000&&lx==0) return atan(y);   /* x=1.0 */
 	m = ((hy>>31)&1)|((hx>>30)&2);	/* 2*sign(x)+sign(y) */
 
     /* when y = 0 */