[Bug 254911] lib/msun/ctrig_test fails if compiled with AVX (-mavx) or any CPUSET enabling AVX

[bugzilla-noreply at freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=254911

--- Comment #2 from Dimitry Andric <dim at FreeBSD.org> ---
OK, for some reason the divide operation in the imaginary part in line 77 gives
rise to FE_INVALID (0x1), but *only* if more advanced CPU instructions are
used:

    41  float complex
    42  ctanhf(float complex z)
    43  {
...
    72          t = tanf(y);
    73          beta = 1.0 + t * t;
    74          s = sinhf(x);
    75          rho = sqrtf(1 + s * s);
    76          denom = 1 + beta * s * s;
    77          return (CMPLXF((beta * rho * s) / denom, t / denom));

For example, one of the test cases has both x and y at zero, and this results
in the values:

t=0, beta=1, s=0, rho=1, denom=1

The real part of the return statement calculates (beta * rho * s) / denom = (1
* 1 * 0) / 1 = 0, but this does *not* change the FP exception flags.

The imaginary part calculates t / denom = 0 / 1, and for some reason this sets
FE_INVALID!

It's tricky to debug this using printfs though, since this also causes
FE_INEXACT, due to some floating point logic in the guts of printf.

Now, as to why compiling for bog-standard x86_64, which *does* use SSE and
SSE2, does not raise FE_INVALID, I'm not sure yet.

-- 
You are receiving this mail because:
You are the assignee for the bug.