UB in various hypot() implementations (left-shifting a negative, number)
Jeff Walden
jwalden at mit.edu
Tue Nov 19 05:55:09 UTC 2019
On 11/16/19 1:14 PM, Steve Kargl wrote:
> Well, clearly, the patch to e_hypotl.c is wrong. It
> clear the significand when t1 = 0 whereas t1 = 1 leaves
> one bit set in the significand. Simply looking at the
> value of t1 under a poor man's debugger shows the
> difference. Adding "printf("%Le %La\n", t1, t1);" after
> the SET_HIGH_WORD gives
>
> 2.962347e-2493 0x1p-8280 <-- t1 = 1
> 0.000000e+00 0x1p-8280 <-- t1 = 0
>
> for hypotl(ldexpl(1.1,-16000), ldexpl(2.1, -16000)).
Hrm. My understanding based on an earlier line in e_hypotl.c was that that change should be identical, but something mildly subtle is going on and my understanding of |long double| is not quite as good as I thought it was. I've removed that file's changes and left just the original two in, then resubmitted the diff.
Jeff
More information about the freebsd-numerics
mailing list