Comparing behavior of test-fesetenv.c on AMD Opterons and Intel Xeons: running FNSTENV on Opteron -- should it zero out __x87.__other?

[NGie Cooper]

Hi,
    I was recently given a bug where one of the tests on a
FreeBSD[-based] OS (it's close-ish to 10.0-RELEASE in some regards)
was failing when running test-fesetenv.c on AMD Opteron processors
using a VMware Hypervisor, because one of the memcmp asserts was being
tripped.
    I did some checking in gdb and it turns out the Intel Xeon
processor was either zero'ing out or leaving alone __x87.__other, and
the AMD Opteron processor was populating __x87.__other with garbage.
Looking at the processor architecture manuals, it seems that the
values are undefined, i.e. can be non-determinate and should be
ignored when running the test, however, I'm not 100% sure if my
analysis is correct. I was wondering if someone who knew a bit more
about amd64 architecture and floating point could comment on this.
    I've included the context from my investigation of the bug at the
bottom of the email.
Thank you!
-NGie

>From lib/msun/amd64/fenv.c:

 40 const fenv_t __fe_dfl_env = {
 41         { 0xffff0000 | __INITIAL_FPUCW__,
 42           0xffff0000,
 43           0xffffffff,
 44           { 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
 45             0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xff, 0xff }
 46         },
 47         __INITIAL_MXCSR__
 48 };

$ grep -r __INITIAL_FPUCW__ sys/
sys/amd64/amd64/fpu.c:  control = __INITIAL_FPUCW__;
sys/amd64/amd64/fpu.c:          if (curpcb->pcb_initial_fpucw !=
__INITIAL_FPUCW__)
sys/amd64/amd64/machdep.c:      pcb->pcb_initial_fpucw = __INITIAL_FPUCW__;
sys/x86/include/fpu.h:#define   __INITIAL_FPUCW__       0x037F
$ grep -r __INITIAL_MXCSR__ sys/
sys/amd64/amd64/fpu.c:  mxcsr = __INITIAL_MXCSR__;
sys/x86/include/fpu.h:#define   __INITIAL_MXCSR__       0x1F80
$

    The numbers in decimal for amd64 are as follows:

0xffff0000 | __INITIAL_FPUCW__ -> 4294902655
0xffff0000 -> 4294901760
0xffffffff -> 4294967295
'\377\377' -> '\xff\xff'
__INITIAL_MXCSR -> 8064

    On Intel, the values between env and FE_DFL_ENV are consistent but
on the system with an AMD Opteron, env.__x87.__other after calling
fesetenv is full of garbage.

 80 #else /* __amd64__ */
 81 typedef struct {
 82         struct {
 83                 __uint32_t      __control;
 84                 __uint32_t      __status;
 85                 __uint32_t      __tag;
 86                 char            __other[16];
 87         } __x87;
 88         __uint32_t              __mxcsr;
 89 } fenv_t;
 90 #endif /* __i386__ */

# cat Makefile
# $FreeBSD$

TESTS=  test-cexp test-conj test-csqrt test-ctrig \
        test-exponential test-fenv test-fma \
        test-fmaxmin test-ilogb test-invtrig test-invctrig \
        test-logarithm test-lrint \
        test-lround test-nan test-nearbyint test-next test-rem test-trig
CFLAGS+= -g -O0 -lm -Wno-unknown-pragmas
CFLAGS+= -std=c99

.PHONY: tests
tests: ${TESTS}
        for p in ${TESTS}; do ${.OBJDIR}/$$p; done

.PHONY: clean
clean:
        -rm -f ${TESTS}
# prove -v test-fenv.t
test-fenv....Assertion failed: (memcmp(&env, FE_DFL_ENV, sizeof(env))
== 0), function test_dfl_env, file test-fenv.c, line 136.
1..8
dubious
        Test returned status 0 (wstat 134, 0x86)
DIED. FAILED tests 1-8
        Failed 8/8 tests, 0.00% okay
Failed Test Stat Wstat Total Fail  Failed  List of Failed
-------------------------------------------------------------------------------
test-fenv.t    0   134     8   16 200.00%  1-8
Failed 1/1 test scripts, 0.00% okay. 8/8 subtests failed, 0.00% okay.
# gdb ./test-fenv
GNU gdb (GDB) 7.7.1 [GDB v7.7.1 for FreeBSD]
Copyright (C) 2014 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-portbld-freebsd11.0".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from ./test-fenv...done.
(gdb) c
The program is not being run.
(gdb) quit
vOneFS-GR3-1# gdb ./test-fenv
GNU gdb (GDB) 7.7.1 [GDB v7.7.1 for FreeBSD]
Copyright (C) 2014 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-portbld-freebsd11.0".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from ./test-fenv...done.
(gdb) r
Starting program: /usr/src/tools/regression/lib/msun/test-fenv
1..8
Assertion failed: (memcmp(&env, FE_DFL_ENV, sizeof(env)) == 0),
function test_dfl_env, file test-fenv.c, line 136.

Program received signal SIGABRT, Aborted.
0x0000000800b842ca in kill () from /lib/libc.so.7
(gdb) frame 3
#3  0x0000000000400ec9 in test_dfl_env () at test-fenv.c:136
136             assert(memcmp(&env, FE_DFL_ENV, sizeof(env)) == 0);
(gdb) p env
$1 = {__x87 = {__control = 4294902655, __status = 4294901760, __tag =
4294967295, __other = "W\224}\200
\000\004\001\250\244D\201;\000\377\377"}, __mxcsr = 8064}
(gdb) p ((fenv_t)__fe_dfl_env)
$2 = {__x87 = {__control = 4294902655, __status = 4294901760, __tag =
4294967295, __other = '\000' <repeats 14 times>, "\377\377"}, __mxcsr
= 8064}

1. http://support.amd.com/TechDocs/26569_APM_v5.pdf
2. http://www.intel.com/Assets/en_US/PDF/manual/253666.pdf