sorting out the echo(1) mess

[Mark Valentine]

I've been working on revamping our rather broken printf(1), and since
echo(1) is related and has fewer issues I thought I'd take a wee detour
and see if there isn't just a tiny chance of getting a concensus on sorting
that out first.

My goal is to try to reach an agreement on what we should fix and what we
should document in the way of standards conformance, backwards and sideways
compatibility, and consistency within our own code base.

A few outstanding issues I can see (please feel free to add any more you're
aware of):

  (A)  echo(1) has a minor compatibility issue (both with traditional
       implementations and current BSD implementations), by way of its
       half-baked \c handling;

  (B)  echo(1) behaves like neither csh(1) nor sh(1) echo, thanks to \c
       and/or -e;

  (C)  sh(1) echo is not POSIX-compliant (though our POSIX 2001 Utility
       Compliance states that it is), due to the -e option; the standard
       has changed wording from "need not" to "do not" to "shall not"
       [support any options] in recent revisions, though the current
       standard *does* special case -n (non-XSI profiles only);

  (D)  our echo(1) and printf(1) %s/%b implementations are inconsistent,
       no matter which way you read the standard.

Here are some proposals to resolve these issues:

  (1)  (a) Remove the \c glitch in echo(1), thus restoring backwards
       compatibility (A) and [partly] consistency (B) - if this is
       unpalatable, then (b) document it as a compatibility bug and
       see (3b);

  (2)  I'd really like to (a) ditch sh(1) echo -e altogether for compliance
       (C) and consistency (B), and tell people to use printf %b to get XSI
       echo behaviour (an informative section of the standard spells this
       out in detail); however, -e has been around too long and the other
       BSD implementations (as well as bash, zsh and so on) have the same
       non-conformance problem, so (b) document it as such instead (but
       otherwise keep echo(1) and sh(1) echo aligned, possibly even sharing
       code (not that there's much of it), as with test(1) and printf(1).
       See (3a).

       NOTE: printf(1) doesn't currently work well enough to serve as XSI
       echo - but watch this space...

  (3)  optional: (a) build an XSI echo(1) variant (strictly conforming) and
       install it somewhere like /usr/xopen/echo; also, if we do (1b) rather
       than (1a), provide (b) a strictly-conforming BSD variant echo(1) -
       without -e - somewhere like /usr/posix/echo.

       I've thrown the idea of /usr/posix and /usr/xopen around in the past
       as ways of controlling the user's environment so far as standards
       conformance in utilities is concerned:

	 PATH=/bin:/usr/bin	 	# native BSD env (backwards compat)
	 PATH=/usr/posix:/bin:/usr/bin	# standard POSIX env + BSD
	 PATH=/usr/xopen:/usr/posix:/bin:/usr/bin # alternate XSI environment

       (As a comparison, consider the following environments in Solaris:

	 PATH=/usr/bin:/usr/ucb		# native System V env (backwards compat)
	 PATH=/usr/xpg4/bin:/usr/bin:/usr/ucb	# standard POSIX+XSI env
	 PATH=/usr/ucb:/usr/bin		# alternate BSD environment

       Well, roughly!)

       /usr/posix is only necessary if we can't/won't make all utilities
       conform to POSIX (even without XSI) - but that can only be achieved
       - as we've already seen with stuff like sed(1) and expr(1) - by
       breaking backwards compatibility (but it's still an option, as
       painful as it is for some of us).

       /usr/xopen is only relevant if we care about any subset of XSI
       extensions which are incompatible with BSD (such as backslash
       escapes in echo(1)!).

       Other options are available, such as moving old compatible versions
       of incompatibly conformance-fixed utilities to a compat bin (I dread
       to see the day we need /usr/ucb on a BSD system, though!), or by way
       of environment variables (fragile) or system tunables (inflexible),
       but I've had experience of all of these variants over the years and
       an approach like the above is the only one that's worked well for me.

  (4)  I'll address printf(1) issues seperately, but just be aware that there
       is a large overlap with echo(1), and so opting against fixing an echo(1)
       conformance issue may adversely affect the ability to fix a consistency
       issue between the two or to fix a printf(1) conformance issue.

Anyway, my main issues right now can be summarised thus:

    * echo(1) can and should be both backwards compatible with traditional
      BSD behaviour and conform to IEEE Std 1003.1-2001; sh(1) echo should
      behave the same.

    * If we can't get away with implementing this, let's make sure the issues
      are documented in the right place.

    * If the default behaviour can't be made standards compliant for whatever
      reason, we should provide a way to create an environment that *does*
      conform to IEEE Std 1003.1-2001 (ISO/IEC 9945:2003), at least for the
      subset of the standard which is relevant to us.

I volunteer both code and documentation (that's the easy bit, it's getting
a concensus that's usually impossible!  Do I get out the grey or the multi-
coloured paint?  ;-)

Comments?  I'll be back in a couple of days to to feel the apathy...

		Cheers,

		Mark.

-- 
"Tigers will do ANYTHING for a tuna fish sandwich."
"We're kind of stupid that way."   *munch* *munch*
  -- <http://www.calvinandhobbes.com>