[RFC] Consistent numeric range for "expr" on all architectures

[Stefan Esser]

Am 29.06.2011 01:06, schrieb Bruce Evans:
> [I changed developers to standards instead of removing it]

Thanks Bruce. I just subcribed to standards@ and was about to move the
thread there, too.

> On Tue, 28 Jun 2011, Stefan Esser wrote:
>> Are there any supported architectures with intmax_t smaller than 64bit?
> 
> There cannot be, since C99 requires long long to be at least 64 bits
> (counting the sign bit) and it requires intmax_t to be capable of
> representing any value of any signed integer type.
> 
> Which checking this, I noticed that:
> - preprocessor arithmetic is done using intmax_t or uintmax_t.  This causes
>   portability problems related to ones for expr -- expressions like
>   ULONG_MAX + ULONG_MAX suddenly started in 1999 giving twice ULONG_MAX
>   instead of ULONG_MAX-1, but only on arches where ULONG_MAX < UINTMAX_MAX.
>   (I use unsigned values in this example to give defined behaviour on
>   overflow, so that the expression ULONG_MAX + ULONG_MAX is not just a bug.
>   expr doesn't have this complication.)
> - C99 doesn't require intmax_t to be the logically longest type.  Thus it
>   permits FreeBSD's rather bizarre implementation of intmax_t being plain
>   long which is logically shorter than long long.
> 
> Other points:
> - `expr -e 10000000000000000000 + 0' (19 zeros) gives "Result too large",
>   but it isn't the result that is too large, but the arg that is too large.
>   This message is strerror(ERANGE) after strtoimax() sets errno to ERANGE.
>   `expr -e 1000000000000000000 \* 10' gives "overflow".  This message is
>   correct, but it is in a different style to strerror() (uncapitalized,
>   and more concise).

The patch that I sent with my first message fixes this. The message is
changed from "Result too large" to "Not a valid integer: %s".

("non-numeric argument" is used in other places and I could adapt to
that, though I prefer to also see which argument was rejected. But I
think that "not a valid integer" better describes the situation.)

Without "-e" the numeric result is "undefined" and no error is signaled,
since there was no test whether the conversion succeeded before I added
it back in 2000.


BTW: The current number parsing is not fully POSIX compliant, since it
allows to write "expr +1 - +1".

This is an artefact of using strtoimax(..., 10) for conversion and
validity checking at the same time (and strtoimax accepts leading "+",
which POSIX forbids). This could be fixed by a simple test of the first
non-blank character of the parameter.

> - `expr 10000000000000000000' (19 or even 119 zeros) gives no error.  It
>   is documented that the arg is parsed as a string in this case, and the
>   documentation for -e doesn't clearly say that -e changes this.  And -e
>   doesn't change this if the arg clearly isn't a number
>   (e.g., if it is 10000000000000000000mumble), or even if it is a non-decimal
>   number (e.g., if is 010, 0x10 or 10.0).  If the arg isn't a decimal integer,
>   then (except for -e on decimal integers), there is an error irrespective
>   of -e when arithmetic is attempted (e.g., adding 0).  The error message
>   for this bogusly says "non-numeric argument" when the arg is numeric but
>   not a decimal integer.

Yes, the reason is the conversion from string to intmax_t just before
the operation. You are right in pointing out that strtoimax can fail for
(out of range) integer numbers.

The error message should probably be changed to "not a valid integer"
instead (or suggest a better form; but the shorter "invalid integer"
seems wrong to me ...).

> - POSIX requires brokenness for bases other than 10, but I wonder if an
>   arg like 0x10 invokes undefined behaviour and thus can be made to
>   work.  (I wanted to use a hex number since I can never remember what
>   INTMAX_MAX is in decimal and wanted to type it in hex for checking
>   the range and overflow errors.)  Allowing hex args causes fewer
>   problems than allowing decimal args larger than INT32_MAX, since
>   they are obviously unportable.  Some FreeBSD utilities, e.g., dd,
>   support hex args and don't worry about POSIX restricting them.

Does POSIX require that expr exits on illegal arguments?

Else we'll have undefined behaviour and might be allowed to add parsing
of hex numbers. I'd be glad to add it (but this would be a different
commit). But octals must not be supported, since 010 must be treated as
decimal number (i.e., we must not just make strtoimax accept other bases
than 10, but must special case for decimal and hex).

> - POSIX unfortunately requires args larger than INT32_MAX to be unportable
>   (to work if longs are longer than 32 bits, else to give undefined (?)
>   behaviour.  For portability there could be a -p switch that limits args
>   to INT32_MAX even if longs are longer than 32 bits.

Well, undefined behaviour can always be to return the correct result ;-)

I'd be willing to add "-p" (effectively just make "-e" the default that
can be overridden).

> - I hope POSIX doesn't require benign overflow.  Thus treating all overflows 
>   as errors is good for portability and doesn't require any switch.

So do I ... But if we agreed on making intmax_t the default and on "-p"
for strict POSIX compliance, then we could of course keep the undetected
overflows for that code path.

Thank you for the comments!

Best regards, STefan