UB in various hypot() implementations (left-shifting a negative, number)

Jeff Walden jwalden at mit.edu
Tue Nov 19 05:55:09 UTC 2019

On 11/16/19 1:14 PM, Steve Kargl wrote:
> Well, clearly, the patch to e_hypotl.c is wrong.  It
> clear the significand when t1 = 0 whereas t1 = 1 leaves
> one bit set in the significand.  Simply looking at the
> value of t1 under a poor man's debugger shows the 
> difference.  Adding "printf("%Le %La\n", t1, t1);" after
> the SET_HIGH_WORD gives
> 2.962347e-2493 0x1p-8280  <-- t1 = 1
> 0.000000e+00 0x1p-8280    <-- t1 = 0
> for hypotl(ldexpl(1.1,-16000), ldexpl(2.1, -16000)).

Hrm.  My understanding based on an earlier line in e_hypotl.c was that that change should be identical, but something mildly subtle is going on and my understanding of |long double| is not quite as good as I thought it was.  I've removed that file's changes and left just the original two in, then resubmitted the diff.


More information about the freebsd-numerics mailing list