Fwd: [RFC] Consistent numeric range for "expr" on all architectures

[Jilles Tjoelker]

On Thu, Jun 30, 2011 at 01:42:08PM -0400, David Schultz wrote:
> > > Overflow checking is a separate concern, and one that is more
> > > likely to cause problems.  I'd be more careful about changing the
> > > default behavior there, because some scripts might rely on modular
> > > arithmetic without overflow checks.

> > You cannot portably rely in overflow, since you have no guarantee that
> > overflow occurs at a specific boundary.

> My point was that some scripts might rely on the lack of overflow
> checking anyway.  This is a separate and potentially more
> problematic change.  But I'm not trying to argue that it's
> necessarily a bad idea.

tools/regression/bin/sh/expansion/arith11.0 depends on two's complement
overflow behaviour, to test dividing the smallest integer by -1.

> > > Another thing that is likely to cause confusion is expr(1)
> > > behaving differently from the shell builtin, and differently from
> > > the shell's arithmetic.  Above all else, I suggest trying to make
> > > everything consistent...perhaps talk to jilles@ about the shell
> > > side of things.

> > My change in 2000 made the shell builtin do 64bit arithmetic as well (in
> > fact, the shell build just included the expr.y file ...).

As far as I can see from the history, there has never been an expr(1)
builtin in FreeBSD. The commented line in builtins.def is deceiving: it
was for Almquist's combined expr and test (which cannot possibly work
reliably and has never been enabled in FreeBSD) but test and [ were
removed from the line when a test(1) builtin was added.

There is a let builtin, with an alias "exp" in 8.x and older, but this
simply concatenates its arguments and evaluates the result as an
arithmetic expression $((...)). I strongly discourage using "let".

> I don't know the state of the shell these days.  Didn't jilles@
> change the shell to do that more recently?  My point here is that
> working to make both the shell and expr behave identically would
> be ideal.

Shell arithmetic has been intmax_t for a while.

At least in all versions with intmax_t, the smallest integer cannot be
specified as a literal. This is because the minus sign is not part of
the literal and the absolute value of the smallest integer (like
9223372036854775808) does not fit in an intmax_t and is
silently clipped (for example to 9223372036854775807).

This problem does not occur when the number is stored in a variable and
the variable is used in the expression without a dollar sign. The
smallest integer will be processed correctly in that case as the whole
string is passed to strtoimax().

The new arithmetic code in 9.0 has brought some subtle changes. If a
variable is used in an expression without a dollar sign, out of range
values are now errors (previously such values were silently clipped).
Apparently ports and other scripts do not mind this check. Regular
literals are still silently clipped.

This usage of strtoimax() also means that things like
  echo $(($(printf "%#x" -1) ))
or
  v=$(printf "%#x" -1); echo $((v))
do not work as might be expected.

A different overflow behaviour for arithmetic operations might be
dangerous, breaking ports and/or other scripts.

Also note that errors in expr(1) and shell arithmetic are handled rather
differently. The former are usually ignored (though an error message
will probably be printed) while the latter will almost always cause the
shell to stop executing the subshell or script.

-- 
Jilles Tjoelker