standards/130067: Wrong numeric limits in system headers?

[Bruce Evans]

The following reply was made to PR standards/130067; it has been noted by GNATS.

From: Bruce Evans <brde at optusnet.com.au>
To: Vaclav Haisman <v.haisman at sh.cvut.cz>
Cc: freebsd-gnats-submit at FreeBSD.org, freebsd-standards at FreeBSD.org,
        imp at FreeBSD.org
Subject: Re: standards/130067: Wrong numeric limits in system headers?
Date: Wed, 31 Dec 2008 22:26:22 +1100 (EST)

 On Tue, 30 Dec 2008, Vaclav Haisman wrote:
 
 > There seems to be a problem with definition of long double limits on FreeBSD i386/6.x.
 >
 > shell::wilx:~/packed_vector> echo | g++ -dD -E -  | sort | grep LDBL_MAX
 > #define __LDBL_MAX_10_EXP__ 4932
 > #define __LDBL_MAX_EXP__ 16384
 > #define __LDBL_MAX__ 1.1897314953572316e+4932L
 >
 > shell::wilx:~/packed_vector> fgrep -rn LDBL_MAX /usr/include
 > [...]
 > /usr/include/machine/float.h:75:#define LDBL_MAX_EXP    16384
 > /usr/include/machine/float.h:76:#define LDBL_MAX        1.1897314953572317650E+4932L
 > /usr/include/machine/float.h:77:#define LDBL_MAX_10_EXP 4932
 > /usr/include/float.h:75:#define LDBL_MAX_EXP    16384
 > /usr/include/float.h:76:#define LDBL_MAX        1.1897314953572317650E+4932L
 > /usr/include/float.h:77:#define LDBL_MAX_10_EXP 4932
 >
 > Notice the difference in definition of LDBL_MAX, the values in system
 > headers are tiny bit larger than that defined by GCC itself.
 > ...
 > machine/float.h:76:#define LDBL_MAX        1.1897314953572317650E+4932L
 > float.h:76:#define LDBL_MAX                1.1897314953572317650E+4932L
 > GCC: __LDBL_MAX__                          1.1897314953572316e+4932L
 
 This has never worked.  However, since gcc became aware of the limited
 precision of long doubles under FreeBSD a few years ago, it shouldn't
 be as broken as it is.  Gcc defines __LDBL_MAX__ to have the correct
 value (1-2^-53)*2^1024 (rounded to 17 digits, which is enough for the
 actual precision of 53 bits), while FreeBSD defines LDBL_MAX as
 (1-2^-64)*2^1024 (rounded to 20 digits, which is perhaps not quite
 enough for the non-actual precision of 64 bits (either this 20 or
 DECIMAL_DIG's value of 21 is dubious).  The wrong value in FreeBSD
 may have worked accidentally a few years ago, but now hit has no
 chance of working:
 - a few years ago: gcc didn't know about FreeBSD's limited precision,
    so it evaluated the constant in 64-bit precision and got precisely
    (1-2^-64)*2^1024 (only the decimal constant is imprecise).  Static
    initialization to this value preserved the value.  Actual use of
    the value caused it to be rounded to 53 bits.  I think that still
    made it Infinity in most cases.
 - now: gcc evaluates it in 53-bit precision and gets Infinity for it
    consistently.
 
 Many other long double constants in FreeBSD's <float.h> have never
 worked, and are much further from neing correct, but are fixed in
 gcc:
 
 From gcc4.2 -E -dM:
 % #define __LDBL_MAX__ 1.1897314953572316e+4932L
 % #define __LDBL_MAX_EXP__ 16384
 % #define __LDBL_HAS_INFINITY__ 1
 % #define __LDBL_MIN__ 3.3621031431120935e-4932L
 % #define __LDBL_HAS_QUIET_NAN__ 1
 % #define __LDBL_HAS_DENORM__ 1
 % #define __LDBL_EPSILON__ 2.2204460492503131e-16L
 % #define __LDBL_DIG__ 15
 % #define __LDBL_MANT_DIG__ 53
 % #define __LDBL_MIN_EXP__ (-16381)
 % #define __LDBL_MAX_10_EXP__ 4932
 % #define __LDBL_DENORM_MIN__ 7.4653686412953080e-4948L
 % #define __LDBL_MIN_10_EXP__ (-4931)
 
 From float.h:
 % #define LDBL_MANT_DIG	64
 
 Wrong; should be 53.  All the other errors are derived from this
 (see the definitions of FLT_* for the derivations -- p must be 53
 but is 64).
 
 % #define LDBL_EPSILON	1.0842021724855044340E-19L
 
 Wrong: too small by a factor of 2^13.
 
 % #define LDBL_DIG	18
 
 Wrong: unnecessarily large (fairly harmless).
 
 % #define LDBL_MIN_EXP	(-16381)
 
 Correct.
 
 % #define LDBL_MIN	3.3621031431120935063E-4932L
 
 Correct (just has more precision than needed for rounding to 53 bits).
 
 % #define LDBL_MIN_10_EXP	(-4931)
 % #define LDBL_MAX_EXP	16384
 
 Correct.
 
 % #define LDBL_MAX	1.1897314953572317650E+4932L
 
 Wrong: see above.
 
 % #define LDBL_MAX_10_EXP	4932
 
 Correct.
 
 Bruce