standards/56906: Several math(3) functions fail to set errno on a domain error

[Bruce Evans]

On Tue, 16 Sep 2003, Stefan Farfeleder wrote:

> >Synopsis:       Several math(3) functions fail to set errno on a domain error

This is intentional.

> >Description:
> ISO C90 says that errno must be set to EDOM if a domain error occurs.

As you probably know, C99 doesn't require this.  FreeBSD, or at least I,
decided not to support C90's relatively feeble and broken specification
of floating point and wait for C99 to get it right.  The wait is long
over and the nonstandardness is now a little different.

As you may know, this existence of errno is basically a bug.  It is
especially broken as designed for math functions because requiring the
side effect of setting it breaks things like removing sqrt(x) (for x
a loop consant) from loops unless the compiler is very clever about
this side effect.  The compiler may also need to be clever about the
side effect of possibly setting IEEE exception flags for sqrt(x), but
the exception flags are sticky and low-level (unlike errno), so it
doesn't need to be so clever.  I believe that C99 relaxed the requirement
on setting errno to simplify such optimization.

The current nonstandardness is mainly related to setting the exception
flags, and many unimplemented functions.  The Sun parts of fdlibm
attempt to set the exception flags, but were often defeated by invalid
compile-time constant folding last time I checked (long ago).  The
assembler parts are not very careful about the exception flags.

> The cases where a domain error definitely has to occur are:
>
> acos(x)		if |x| > 1,
> asin(x)		if |x| > 1,
> log(x)		if x < 0,
> log10(x)	if x < 0,
> sqrt(x)		if x < 0 and
> pow(x, y)	if x < 0 && y not an integer or if x == 0 && y <= 0.
>
> FreeBSD's libm fails to set errno on each of these cases even though the
> man pages of acos, asin and sqrt claim conformance to C89/90.

Oops.  The man pages even claim to set errno.

> >Fix:
> Compiling msun with -D_POSIX_MODE fixes the setting of errno.  I have no
> idea if this breaks other parts of libm though; there don't seem to be
> any regression tests for libm.

It is safe enough for strict C90, but it adds overhead.  For unstrict
C90 == C90 less errno plus IEEEish extensions, and for C99 with IEEE
support, it breaks returning NaNs for domain errors.  E.g., it causes
acos(2.0) to return the preposterous nonstandard value 0 with errno
set to EDOM instead of a relatively standard non-value "NaN" with errno
unchanged.  See k_standard.c.  It should not adjust the value except
possibly in the (_LIB_VERSION == _SVID_) case, but k_standard.c is
not passed the original value and I was too lazy to fix its interface.

Bruce