git: daf917daba9c - stable/14 - Merge one true awk from 2024-01-22 for the Awk Second Edition support

From: Warner Losh <imp_at_FreeBSD.org>
Date: Tue, 12 Mar 2024 14:49:09 UTC
The branch stable/14 has been updated by imp:

URL: https://cgit.FreeBSD.org/src/commit/?id=daf917daba9c41c2e11dd5470e632a6ebf4f38e1

commit daf917daba9c41c2e11dd5470e632a6ebf4f38e1
Author:     Warner Losh <imp@FreeBSD.org>
AuthorDate: 2024-02-28 15:16:16 +0000
Commit:     Warner Losh <imp@FreeBSD.org>
CommitDate: 2024-03-12 04:53:18 +0000

    Merge one true awk from 2024-01-22 for the Awk Second Edition support
    
    This brings in Unicode support, CSV support and a number of bug fixes.
    They are described in _The AWK Programming Language_, Second Edition, by
    Al Aho, Brian Kernighan, and Peter Weinberger (Addison-Wesley, 2024,
    ISBN-13 978-0138269722, ISBN-10 0138269726).
    
    Sponsored by:           Netflix
    
    (cherry picked from commit f32a6403d34654ac6e61182d09abb5e85850e1ee)
---
 contrib/one-true-awk/FIXES                         | 1429 ++------------------
 contrib/one-true-awk/FIXES.1e                      | 1429 ++++++++++++++++++++
 contrib/one-true-awk/README.md                     |   80 +-
 contrib/one-true-awk/awk.1                         |   34 +-
 contrib/one-true-awk/awk.h                         |   23 +-
 contrib/one-true-awk/awkgram.y                     |   49 +-
 contrib/one-true-awk/b.c                           |  409 ++++--
 contrib/one-true-awk/bugs-fixed/REGRESS            |    8 +-
 .../one-true-awk/bugs-fixed/getline-corruption.awk |    5 +
 .../one-true-awk/bugs-fixed/getline-corruption.in  |    1 +
 .../one-true-awk/bugs-fixed/getline-corruption.ok  |    1 +
 contrib/one-true-awk/bugs-fixed/matchop-deref.awk  |   11 +
 contrib/one-true-awk/bugs-fixed/matchop-deref.bad  |    2 +
 contrib/one-true-awk/bugs-fixed/matchop-deref.in   |    1 +
 contrib/one-true-awk/bugs-fixed/matchop-deref.ok   |    2 +
 .../one-true-awk/bugs-fixed/missing-precision.ok   |    2 +
 contrib/one-true-awk/bugs-fixed/negative-nf.ok     |    2 +
 contrib/one-true-awk/bugs-fixed/pfile-overflow.ok  |    4 +
 contrib/one-true-awk/bugs-fixed/rstart-rlength.awk |   10 +
 contrib/one-true-awk/bugs-fixed/rstart-rlength.ok  |    4 +
 contrib/one-true-awk/bugs-fixed/system-status.awk  |   19 +
 contrib/one-true-awk/bugs-fixed/system-status.bad  |    3 +
 contrib/one-true-awk/bugs-fixed/system-status.ok   |    3 +
 contrib/one-true-awk/bugs-fixed/system-status.ok2  |    3 +
 .../one-true-awk/bugs-fixed/unicode-fs-rs-1.awk    |    6 +
 contrib/one-true-awk/bugs-fixed/unicode-fs-rs-1.in |    2 +
 contrib/one-true-awk/bugs-fixed/unicode-fs-rs-1.ok |    5 +
 .../one-true-awk/bugs-fixed/unicode-fs-rs-2.awk    |    7 +
 contrib/one-true-awk/bugs-fixed/unicode-fs-rs-2.in |    2 +
 contrib/one-true-awk/bugs-fixed/unicode-fs-rs-2.ok |    4 +
 .../one-true-awk/bugs-fixed/unicode-null-match.awk |    6 +
 .../one-true-awk/bugs-fixed/unicode-null-match.bad |    1 +
 .../one-true-awk/bugs-fixed/unicode-null-match.ok  |    1 +
 contrib/one-true-awk/lex.c                         |   60 +-
 contrib/one-true-awk/lib.c                         |  158 ++-
 contrib/one-true-awk/main.c                        |   23 +-
 contrib/one-true-awk/makefile                      |    8 +-
 contrib/one-true-awk/maketab.c                     |    4 +-
 contrib/one-true-awk/parse.c                       |    2 +-
 contrib/one-true-awk/proto.h                       |   10 +-
 contrib/one-true-awk/run.c                         |  942 +++++++++----
 contrib/one-true-awk/testdir/Compare.tt            |    2 +-
 contrib/one-true-awk/testdir/REGRESS               |    2 +-
 contrib/one-true-awk/testdir/T.argv                |    6 +
 contrib/one-true-awk/testdir/T.csv                 |   80 ++
 contrib/one-true-awk/testdir/T.flags               |    5 +-
 contrib/one-true-awk/testdir/T.misc                |   20 +
 contrib/one-true-awk/testdir/T.overflow            |    2 +
 contrib/one-true-awk/testdir/T.split               |    1 +
 contrib/one-true-awk/testdir/T.utf                 |  194 +++
 contrib/one-true-awk/testdir/T.utfre               |  234 ++++
 contrib/one-true-awk/testdir/tt.15                 |    2 +-
 contrib/one-true-awk/tran.c                        |   26 +-
 53 files changed, 3525 insertions(+), 1824 deletions(-)

diff --git a/contrib/one-true-awk/FIXES b/contrib/one-true-awk/FIXES
index 8a2befccd624..a043b356fafa 100644
--- a/contrib/one-true-awk/FIXES
+++ b/contrib/one-true-awk/FIXES
@@ -22,1354 +22,87 @@ ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF
 THIS SOFTWARE.
 ****************************************************************/
 
-This file lists all bug fixes, changes, etc., made since the AWK book
-was sent to the printers in August, 1987.
+This file lists all bug fixes, changes, etc., made since the 
+second edition of the AWK book was published in September 2023.
 
-July 27, 2021:
-	As per IEEE Std 1003.1-2008, -F "str" is now consistent with
-	-v FS="str" when str is null. Thanks to Warner Losh.
-
-July 24, 2021:
-	Fix readrec's definition of a record. This fixes an issue
-	with NetBSD's RS regular expression support that can cause
-	an infinite read loop. Thanks to Miguel Pineiro Jr.
-
-	Fix regular expression RS ^-anchoring. RS ^-anchoring needs to
-	know if it is reading the first record of a file. This change
-	restores a missing line that was overlooked when porting NetBSD's
-	RS regex functionality. Thanks to Miguel Pineiro Jr.
-
-	Fix size computation in replace_repeat() for special case
-	REPEAT_WITH_Q. Thanks to Todd C. Miller.
-
-February 15, 2021:
-	Small fix so that awk will compile again with g++. Thanks to
+Jan 22, 2024:
+	Restore the ability to compile with g++. Thanks to
 	Arnold Robbins.
 
-January 06, 2021:
-	Fix a decision bug with trailing stuff in lib.c:is_valid_number
-	after recent changes. Thanks to Ozan Yigit.
-
-December 18, 2020:
-	Fix problems converting inf and NaN values in lib.c:is_valid_number.
-	Enhance number to string conversion to do the right thing for
-	NaN and inf values.  Things are now pretty much the same as in
-	gawk.  (Found a gawk bug while we're at it.) Added a torture
-	test for these values.  Thanks to Arnold Robbins.  Allows closing
-	of PR #101.
-
-December 15, 2020:
-	Merge PR #99, which gets the right header for strcasecmp.
-	Thanks to GitHub user michaelforney.
-
-December 8, 2020:
-	Merge PR #98: Disallow hex data. Allow only +nan, -nan,
-	+inf, -inf (case independent) to give NaN and infinity values.
-	Improve things so that string to double conversion is only
-	done once, yielding something of a speedup.  This obviate
-	PR #95. Thanks to Arnold Robbins.
-
-December 3, 2020:
-	Fix to argument parsing to avoid printing spurious newlines.
-	Thanks to Todd Miller. Merges PR #97.
-
-October 13, 2020:
-	Add casts before all the calls to malloc/calloc/realloc in order
-	to get it to compile with g++. Thanks to Arnold Robbins.
-
-August 16, 2020:
-	Additional fixes for DJGPP. Thanks to Eli Zaretskii for
-	the testing.
-
-August 7, 2020:
-	Merge PR #93, which adds casts to (void*) for debug prints
-	using the %p format specifier. Thanks to GitHub user YongHaoWu
-	("Chris") for the fixes.
-
-August 4, 2020:
-	In run.c, use non-restartable multibyte routines to attain
-	portability to DJGPP. Should fix Issue 92. Thanks to Albert Wik
-	for the report and to Todd Miller for the suggested fix.
-
-July 30, 2020:
-	Merge PRs 88-91 which fix small bugs. Thanks to Todd Miller and
-	Tim van der Molen for the fixes.
-
-	In order to make life easier, we move exclusively to bison
-	as the parser generator.
-
-July 2, 2020:
-	Merge PRs 85 and 86 which fix regressions. Thanks to
-	Tim van der Molen for the fixes.
-
-June 25, 2020:
-	Merge PRs 82 and 84. The latter fixes issue #83. Thanks to
-	Todd Miller and awkfan77.
-
-June 12, 2020:
-	Clear errno before calling errcheck to avoid any spurious errors
-	left over from previous calls that may have set it. Thanks to
-	Todd Miller for the fix, from PR #80.
-
-	Fix Issue #78 by allowing \r to follow floating point numbers in
-	lib.c:is_number. Thanks to GitHub user ajcarr for the report
-	and to Arnold Robbins for the fix.
-
-June 5, 2020:
-	In fldbld(), make sure that inputFS is set before trying to
-	use it. Thanks to  Steffen Nurpmeso <steffen@sdaoden.eu>
-	for the report.
-
-May 5, 2020:
-	Fix checks for compilers that can handle noreturn. Thanks to
-	GitHub user enh-google for pointing it out. Closes Issue #79.
-
-April 16, 2020:
-	Handle old compilers that don't support C11 (for noreturn).
-	Thanks to Arnold Robbins.
-
-April 5, 2020:
-	Use <stdnoreturn.h> and noreturn instead of GCC attributes.
-	Thanks to GitHub user awkfan77. Closes PR #77.
-
-February 28, 2020:
-	More cleanups from Christos Zoulas: notably backslash continuation
-	inside strings removes the newline and a fix for RS = "^a".
-	Fix for address sanitizer-found problem. Thanks to GitHub user
-	enh-google.
-
-February 19, 2020:
-	More small cleanups from Christos Zoulas.
-
-February 18, 2020:
-	Additional cleanups from Christos Zoulas. It's no longer necessary
-	to use the -y flag to bison.
-
-February 6, 2020:
-	Additional small cleanups from Christos Zoulas. awk is now
-	a little more robust about reporting I/O errors upon exit.
-
-January 31, 2020:
-	Merge PR #70, which avoids use of variable length arrays. Thanks
-	to GitHub user michaelforney.  Fix issue #60 ({0} in interval
-	expressions doesn't work).  Also get all tests working again.
-	Thanks to Arnold Robbins.
-
-January 24, 2020:
-	A number of small cleanups from Christos Zoulas.  Add the close
-	on exec flag to files/pipes opened for redirection; courtesy of
-	Arnold Robbins.
-
-January 19, 2020:
-	If POSIXLY_CORRECT is set in the environment, then sub and gsub
-	use POSIX rules for multiple backslashes.  This fixes Issue #66,
-	while maintaining backwards compatibility.
-
-January 9, 2020:
-	Input/output errors on closing files are now fatal instead of
-	mere warnings. Thanks to Martijn Dekker <martijn@inlv.org>.
-
-January 5, 2020:
-	Fix a bug in the concatentation of two string constants into
-	one done in the grammar.  Fixes GitHub issue #61.  Thanks
-	to GitHub user awkfan77 for pointing out the direction for
-	the fix.  New test T.concat added to the test suite.
-	Fix a few memory leaks reported by valgrind, as well.
-
-December 27, 2019:
-	Fix a bug whereby a{0,3} could match four a's.  Thanks to
-	"Anonymous AWK fan" for the report.
-
-December 11, 2019:
-	Further printf-related fixes for 32 bit systems.
-	Thanks again to Christos Zoulas.
-
-December 8, 2019:
-	Fix the return value of sprintf("%d") on 32 bit systems.
-	Thanks to Jim Lowe for the report and to Christos Zoulas
-	for the fix.
-
-November 10, 2019:
-	Convert a number of Boolean integer variables into
-	actual bools. Convert compile_time variable into an
-	enum and simplify some of the related code.  Thanks
-	to Arnold Robbins.
-
-November 8, 2019:
-	Fix from Ori Bernstein to get UTF-8 characters instead of
-	bytes when FS = "".  This is currently the only bit of
-	the One True Awk that understands multibyte characters.
-	From Arnold Robbins, apply some cleanups in the test suite.
-
-October 25, 2019:
-	More fixes and cleanups from NetBSD, courtesy of Christos
-	Zoulas. Merges PRs 54 and 55.
-
-October 24, 2019:
-	Import second round of code cleanups from NetBSD. Much thanks
-	to Christos Zoulas (GitHub user zoulasc). Merges PR 53.
-	Add an optimization for string concatenation, also from
-	Christos.
-
-October 17, 2019:
-	Import code cleanups from NetBSD. Much thanks to Christos
-	Zoulas (GitHub user zoulasc). Merges PR 51.
-
-October 6, 2019:
-	Import code from NetBSD awk that implements RS as a regular
-	expression.
-
-September 10, 2019:
-	Fixes for various array / memory overruns found via gcc's
-	-fsanitize=unknown. Thanks to Alexander Richardson (GitHub
-	user arichardson). Merges PRs 47 and 48.
-
-July 28, 2019:
-	Import grammar optimization from NetBSD: Two string constants
-	concatenated together get turned into a single string.
-
-July 26, 2019:
-	Support POSIX-specified C-style escape sequences "\a" (alarm)
-	and "\v" (vertical tab) in command line arguments and regular
-	expressions, further to the support for them in strings added on
-	Apr 9, 1989. These now no longer match as literal "a" and "v"
-	characters (as they don't on other awk implementations).
-	Thanks to Martijn Dekker.
-
-July 17, 2019:
-	Pull in a number of code cleanups and minor fixes from
-	Warner Losh's bsd-ota branch.  The only user visible change
-	is the use of random(3) as the random number generator.
-	Thanks to Warner Losh for collecting all these fixes in
-	one easy place to get them from.
-
-July 16, 2019:
-	Fix field splitting to use FS value as of the time a record
-	was read or assigned to.  Thanks to GitHub user Cody Mello (melloc)
-	for the fix. (Merged from his branch, via PR #42.) Updated
-	testdir/T.split per said PR as well.
-
-June 24, 2019:
-	Extract awktest.tar into testdir directory. Add some very
-	simple mechanics to the makefile for running the tests and
-	for cleaning up. No changes to awk itself.
-
-June 17, 2019:
-	Disallow deleting SYMTAB and its elements, which creates
-	use-after-free bugs. Thanks to GitHub user Cody Mello (melloc)
-	for the fix. (Merged from PR #43.)
-
-June 5, 2019:
-	Allow unmatched right parenthesis in a regular expression to
-	be treated literally. Fixes Issue #40. Thanks to GitHub user
-	Warner Losh (bsdimp) for the report. Thanks to Arnold Robbins
-	for the fix.
-
-May 29,2019:
-	Fix check for command line arguments to no longer require that
-	first character after '=' not be another '='. Reverts change of
-	August 11, 1989. Thanks to GitHub user Jamie Landeg Jones for
-	pointing out the issue; from Issue #38.
-
-Apr 7, 2019:
-	Update awktest.tar(p.50) to use modern options to sort. Needed
-	for Android development. Thanks to GitHub user mohd-akram (Mohamed
-	Akram).  From Issue #33.
-
-Mar 12, 2019:
-	Added very simplistic support for cross-compiling in the
-	makefile.  We are NOT going to go in the direction of the
-	autotools, though.  Thanks to GitHub user nee-san for
-	the basic change. (Merged from PR #34.)
-
-Mar 5, 2019:
-	Added support for POSIX-standard interval expressions (a.k.a.
-	bounds, a.k.a. repetition expressions) in regular expressions,
-	backported (via NetBSD) from Apple awk-24 (20070501).
-	Thanks to Martijn Dekker <martijn@inlv.org> for the port.
-	(Merged from PR #30.)
-
-Mar 3, 2019:
-	Merge PRs as follows:
-	#12: Avoid undefined behaviour when using ctype(3) functions in
-	     relex(). Thanks to GitHub user iamleot.
-	#31: Make getline handle numeric strings, and update FIXES. Thanks
-	     to GitHub user arnoldrobbins.
-	#32: maketab: support build systems with read-only source. Thanks
-	     to GitHub user enh.
-
-Jan 25, 2019:
-	Make getline handle numeric strings properly in all cases.
-	(Thanks, Arnold.)
-
-Jan 21, 2019:
-	Merged a number of small fixes from GitHub pull requests.
-	Thanks to GitHub users Arnold Robbins (arnoldrobbins),
-	Cody Mello (melloc) and Christoph Junghans (junghans).
-	PR numbers: 13-21, 23, 24, 27.
-
-Oct 25, 2018:
-	Added test in maketab.c to prevent generating a proctab entry
-	for YYSTYPE_IS_DEFINED.  It was harmless but some gcc settings
-	generated a warning message.  Thanks to Nan Xiao for report.
-
-Aug 27, 2018:
-	Disallow '$' in printf formats; arguments evaluated in order
-	and printed in order.
-
-	Added some casts to silence warnings on debugging printfs.
-	(Thanks, Arnold.)
-
-Aug 23, 2018:
-        A long list of fixes courtesy of Arnold Robbins,
-        to whom profound thanks.
-
-        1. ofs-rebuild: OFS value used to rebuild the record was incorrect.
-        Fixed August 19, 2014. Revised fix August 2018.
-
-        2. system-status: Instead of a floating-point division by 256, use
-        the wait(2) macros to create a reasonable exit status.
-        Fixed March 12, 2016.
-
-        3. space: Use provided xisblank() function instead of ispace() for
-        matching [[:blank:]].
-
-        4. a-format: Add POSIX standard %a and %A to supported formats. Check
-        at runtime that this format is available.
-
-        5. decr-NF: Decrementing NF did not change $0. This is a decades-old
-        bug. There are interactions with the old and new value of OFS as well.
-        Most of the fix came from the NetBSD awk.
-
-        6. string-conv: String conversions of scalars were sticky.  Once a
-        conversion to string happened, even with OFMT, that value was used until
-        a new numeric value was assigned, even if OFMT differed from CONVFMT,
-        and also if CONVFMT changed.
-
-        7. unary-plus: Unary plus on a string constant returned the string.
-        Instead, it should convert the value to numeric and give that value.
-
-	Also added Arnold's tests for these to awktest.tar as T.arnold.
-
-Aug 15, 2018:
-	fixed mangled awktest.tar (thanks, Arnold), posted all
-	current (very minor) fixes to github / onetrueawk
-
-Jun 7, 2018:
-	(yes, a long layoff)
-	Updated some broken tests (beebe.tar, T.lilly)
-	[thanks to Arnold Robbins]
-
-Mar 26, 2015:
-	buffer overflow in error reporting; thanks to tobias ulmer
-	and john-mark gurney for spotting it and the fix.
-
-Feb 4, 2013:
-	cleaned up a handful of tests that didn't seem to actually
-	test for correct behavior: T.latin1, T.gawk.
-
-Jan 5, 2013:
-	added ,NULL initializer to static Cells in run.c; not really
-	needed but cleaner.  Thanks to Michael Bombardieri.
-
-Dec 20, 2012:
-	fiddled makefile to get correct yacc and bison flags.  pick yacc
-	(linux) or bison (mac) as necessary.
-
-	added  __attribute__((__noreturn__)) to a couple of lines in
-	proto.h, to silence someone's enthusiastic checker.
-
-	fixed obscure call by value bug in split(a[1],a) reported on
-	9fans.  the management of temporary values is just a mess; i
-	took a shortcut by making an extra string copy.  thanks
-	to paul patience and arnold robbins for passing it on and for
-	proposed patches.
-
-	tiny fiddle in setfval to eliminate -0 results in T.expr, which
-	has irritated me for 20+ years.
-
-Aug 10, 2011:
-	another fix to avoid core dump with delete(ARGV); again, many thanks
-	to ruslan ermilov.
-
-Aug 7, 2011:
-	split(s, a, //) now behaves the same as split(s, a, "")
-
-Jun 12, 2011:
-	/pat/, \n /pat/ {...} is now legal, though bad style to use.
-
-	added checks to new -v code that permits -vnospace; thanks to
-	ruslan ermilov for spotting this and providing the patch.
-
-	removed fixed limit on number of open files; thanks to aleksey
-	cheusov and christos zoulos.
-
-	fixed day 1 bug that resurrected deleted elements of ARGV when
-	used as filenames (in lib.c).
-
-	minor type fiddles to make gcc -Wall -pedantic happier (but not
-	totally so); turned on -fno-strict-aliasing in makefile.
-
-May 6, 2011:
-	added #ifdef for isblank.
-	now allows -ffoo as well as -f foo arguments.
-	(thanks, ruslan)
-
-May 1, 2011:
-	after advice from todd miller, kevin lo, ruslan ermilov,
-	and arnold robbins, changed srand() to return the previous
-	seed (which is 1 on the first call of srand).  the seed is
-	an Awkfloat internally though converted to unsigned int to
-	pass to the library srand().  thanks, everyone.
-
-	fixed a subtle (and i hope low-probability) overflow error
-	in fldbld, by adding space for one extra \0.  thanks to
-	robert bassett for spotting this one and providing a fix.
-
-	removed the files related to compilation on windows.  i no
-	longer have anything like a current windows environment, so
-	i can't test any of it.
-
-May 23, 2010:
-	fixed long-standing overflow bug in run.c; many thanks to
-	nelson beebe for spotting it and providing the fix.
-
-	fixed bug that didn't parse -vd=1 properly; thanks to santiago
-	vila for spotting it.
-
-Feb 8, 2010:
-	i give up.  replaced isblank with isspace in b.c; there are
-	no consistent header files.
-
-Nov 26, 2009:
-	fixed a long-standing issue with when FS takes effect.  a
-	change to FS is now noticed immediately for subsequent splits.
-
-	changed the name getline() to awkgetline() to avoid yet another
-	name conflict somewhere.
-
-Feb 11, 2009:
-	temporarily for now defined HAS_ISBLANK, since that seems to
-	be the best way through the thicket.  isblank arrived in C99,
-	but seems to be arriving at different systems at different
-	times.
-
-Oct 8, 2008:
-	fixed typo in b.c that set tmpvec wrongly.  no one had ever
-	run into the problem, apparently.  thanks to alistair crooks.
-
-Oct 23, 2007:
-	minor fix in lib.c: increase inputFS to 100, change malloc
-	for fields to n+1.
-
-	fixed memory fault caused by out of order test in setsval.
-
-	thanks to david o'brien, freebsd, for both fixes.
-
-May 1, 2007:
-	fiddle in makefile to fix for BSD make; thanks to igor sobrado.
-
-Mar 31, 2007:
-	fixed some null pointer refs calling adjbuf.
-
-Feb 21, 2007:
-	fixed a bug in matching the null RE in sub and gsub.  thanks to al aho
-	who actually did the fix (in b.c), and to wolfgang seeberg for finding
-	it and providing a very compact test case.
-
-	fixed quotation in b.c; thanks to Hal Pratt and the Princeton Dante
-	Project.
-
-	removed some no-effect asserts in run.c.
-
-	fiddled maketab.c to not complain about bison-generated values.
-
-	removed the obsolete -V argument; fixed --version to print the
-	version and exit.
-
-	fixed wording and an outright error in the usage message; thanks to igor
-	sobrado and jason mcintyre.
-
-	fixed a bug in -d that caused core dump if no program followed.
-
-Jan 1, 2007:
-	dropped mac.code from makefile; there are few non-MacOSX
-	mac's these days.
-
-Jan 17, 2006:
-	system() not flagged as unsafe in the unadvertised -safe option.
-	found it while enhancing tests before shipping the ;login: article.
-	practice what you preach.
-
-	removed the 9-years-obsolete -mr and -mf flags.
-
-	added -version and --version options.
-
-	core dump on linux with BEGIN {nextfile}, now fixed.
-
-	removed some #ifdef's in run.c and lex.c that appear to no
-	longer be necessary.
-
-Apr 24, 2005:
-	modified lib.c so that values of $0 et al are preserved in the END
-	block, apparently as required by posix.  thanks to havard eidnes
-	for the report and code.
-
-Jan 14, 2005:
-	fixed infinite loop in parsing, originally found by brian tsang.
-	thanks to arnold robbins for a suggestion that started me
-	rethinking it.
-
-Dec 31, 2004:
-	prevent overflow of -f array in main, head off potential error in
-	call of SYNTAX(), test malloc return in lib.c, all with thanks to
-	todd miller.
-
-Dec 22, 2004:
-	cranked up size of NCHARS; coverity thinks it can be overrun with
-	smaller size, and i think that's right.  added some assertions to b.c
-	to catch places where it might overrun.  the RE code is still fragile.
-
-Dec 5, 2004:
-	fixed a couple of overflow problems with ridiculous field numbers:
-	e.g., print $(2^32-1).  thanks to ruslan ermilov, giorgos keramidas
-	and david o'brien at freebsd.org for patches.  this really should
-	be re-done from scratch.
-
-Nov 21, 2004:
-	fixed another 25-year-old RE bug, in split.  it's another failure
-	to (re-)initialize.  thanks to steve fisher for spotting this and
-	providing a good test case.
-
-Nov 22, 2003:
-	fixed a bug in regular expressions that dates (so help me) from 1977;
-	it's been there from the beginning.  an anchored longest match that
-	was longer than the number of states triggered a failure to initialize
-	the machine properly.  many thanks to moinak ghosh for not only finding
-	this one but for providing a fix, in some of the most mysterious
-	code known to man.
-
-	fixed a storage leak in call() that appears to have been there since
-	1983 or so -- a function without an explicit return that assigns a
-	string to a parameter leaked a Cell.  thanks to moinak ghosh for
-	spotting this very subtle one.
-
-Jul 31, 2003:
-	fixed, thanks to andrey chernov and ruslan ermilov, a bug in lex.c
-	that mis-handled the character 255 in input.  (it was being compared
-	to EOF with a signed comparison.)
-
-Jul 29, 2003:
-	fixed (i think) the long-standing botch that included the beginning of
-	line state ^ for RE's in the set of valid characters; this led to a
-	variety of odd problems, including failure to properly match certain
-	regular expressions in non-US locales.  thanks to ruslan for keeping
-	at this one.
-
-Jul 28, 2003:
-	n-th try at getting internationalization right, with thanks to volker
-	kiefel, arnold robbins and ruslan ermilov for advice, though they
-	should not be blamed for the outcome.  according to posix, "."  is the
-	radix character in programs and command line arguments regardless of
-	the locale; otherwise, the locale should prevail for input and output
-	of numbers.  so it's intended to work that way.
-
-	i have rescinded the attempt to use strcoll in expanding shorthands in
-	regular expressions (cclenter).  its properties are much too
-	surprising; for example [a-c] matches aAbBc in locale en_US but abBcC
-	in locale fr_CA.  i can see how this might arise by implementation
-	but i cannot explain it to a human user.  (this behavior can be seen
-	in gawk as well; we're leaning on the same library.)
-
-	the issue appears to be that strcoll is meant for sorting, where
-	merging upper and lower case may make sense (though note that unix
-	sort does not do this by default either).  it is not appropriate
-	for regular expressions, where the goal is to match specific
-	patterns of characters.  in any case, the notations [:lower:], etc.,
-	are available in awk, and they are more likely to work correctly in
-	most locales.
-
-	a moratorium is hereby declared on internationalization changes.
-	i apologize to friends and colleagues in other parts of the world.
-	i would truly like to get this "right", but i don't know what
-	that is, and i do not want to keep making changes until it's clear.
-
-Jul 4, 2003:
-	fixed bug that permitted non-terminated RE, as in "awk /x".
-
-Jun 1, 2003:
-	subtle change to split: if source is empty, number of elems
-	is always 0 and the array is not set.
-
-Mar 21, 2003:
-	added some parens to isblank, in another attempt to make things
-	internationally portable.
-
-Mar 14, 2003:
-	the internationalization changes, somewhat modified, are now
-	reinstated.  in theory awk will now do character comparisons
-	and case conversions in national language, but "." will always
-	be the decimal point separator on input and output regardless
-	of national language.  isblank(){} has an #ifndef.
-
-	this no longer compiles on windows: LC_MESSAGES isn't defined
-	in vc6++.
-
-	fixed subtle behavior in field and record splitting: if FS is
-	a single character and RS is not empty, \n is NOT a separator.
-	this tortuous reading is found in the awk book; behavior now
-	matches gawk and mawk.
-
-Dec 13, 2002:
-	for the moment, the internationalization changes of nov 29 are
-	rolled back -- programs like x = 1.2 don't work in some locales,
-	because the parser is expecting x = 1,2.  until i understand this
-	better, this will have to wait.
-
-Nov 29, 2002:
-	modified b.c (with tiny changes in main and run) to support
-	locales, using strcoll and iswhatever tests for posix character
-	classes.  thanks to ruslan ermilov (ru@freebsd.org) for code.
-	the function isblank doesn't seem to have propagated to any
-	header file near me, so it's there explicitly.  not properly
-	tested on non-ascii character sets by me.
-
-Jun 28, 2002:
-	modified run/format() and tran/getsval() to do a slightly better
-	job on using OFMT for output from print and CONVFMT for other
-	number->string conversions, as promised by posix and done by
-	gawk and mawk.  there are still places where it doesn't work
-	right if CONVFMT is changed; by then the STR attribute of the
-	variable has been irrevocably set.  thanks to arnold robbins for
-	code and examples.
-
-	fixed subtle bug in format that could get core dump.  thanks to
-	Jaromir Dolecek <jdolecek@NetBSD.org> for finding and fixing.
-	minor cleanup in run.c / format() at the same time.
-
-	added some tests for null pointers to debugging printf's, which
-	were never intended for external consumption.  thanks to dave
-	kerns (dkerns@lucent.com) for pointing this out.
-
-	GNU compatibility: an empty regexp matches anything (thanks to
-	dag-erling smorgrav, des@ofug.org).  subject to reversion if
-	this does more harm than good.
-
-	pervasive small changes to make things more const-correct, as
-	reported by gcc's -Wwrite-strings.  as it says in the gcc manual,
-	this may be more nuisance than useful.  provoked by a suggestion
-	and code from arnaud desitter, arnaud@nimbus.geog.ox.ac.uk
-
-	minor documentation changes to note that this now compiles out
-	of the box on Mac OS X.
-
-Feb 10, 2002:
-	changed types in posix chars structure to quiet solaris cc.
-
-Jan 1, 2002:
-	fflush() or fflush("") flushes all files and pipes.
-
-	length(arrayname) returns number of elements; thanks to
-	arnold robbins for suggestion.
-
-	added a makefile.win to make it easier to build on windows.
-	based on dan allen's buildwin.bat.
-
-Nov 16, 2001:
-	added support for posix character class names like [:digit:],
-	which are not exactly shorter than [0-9] and perhaps no more
-	portable.  thanks to dag-erling smorgrav for code.
-
-Feb 16, 2001:
-	removed -m option; no longer needed, and it was actually
-	broken (noted thanks to volker kiefel).
-
-Feb 10, 2001:
-	fixed an appalling bug in gettok: any sequence of digits, +,-, E, e,
-	and period was accepted as a valid number if it started with a period.
-	this would never have happened with the lex version.
-
-	other 1-character botches, now fixed, include a bare $ and a
-	bare " at the end of the input.
-
-Feb 7, 2001:
-	more (const char *) casts in b.c and tran.c to silence warnings.
-
-Nov 15, 2000:
-	fixed a bug introduced in august 1997 that caused expressions
-	like $f[1] to be syntax errors.  thanks to arnold robbins for
-	noticing this and providing a fix.
-
-Oct 30, 2000:
-	fixed some nextfile bugs: not handling all cases.  thanks to
-	arnold robbins for pointing this out.  new regressions added.
-
-	close() is now a function.  it returns whatever the library
-	fclose returns, and -1 for closing a file or pipe that wasn't
-	opened.
-
-Sep 24, 2000:
-	permit \n explicitly in character classes; won't work right
-	if comes in as "[\n]" but ok as /[\n]/, because of multiple
-	processing of \'s.  thanks to arnold robbins.
-
-July 5, 2000:
-	minor fiddles in tran.c to keep compilers happy about uschar.
-	thanks to norman wilson.
-
-May 25, 2000:
-	yet another attempt at making 8-bit input work, with another
-	band-aid in b.c (member()), and some (uschar) casts to head
-	off potential errors in subscripts (like isdigit).  also
-	changed HAT to NCHARS-2.  thanks again to santiago vila.
-
-	changed maketab.c to ignore apparently out of range definitions
-	instead of halting; new freeBSD generates one.  thanks to
-	jon snader <jsnader@ix.netcom.com> for pointing out the problem.
-
-May 2, 2000:
-	fixed an 8-bit problem in b.c by making several char*'s into
-	unsigned char*'s.  not clear i have them all yet.  thanks to
-	Santiago Vila <sanvila@unex.es> for the bug report.
-
-Apr 21, 2000:
-	finally found and fixed a memory leak in function call; it's
-	been there since functions were added ~1983.  thanks to
-	jon bentley for the test case that found it.
-
-	added test in envinit to catch environment "variables" with
-	names beginning with '='; thanks to Berend Hasselman.
-
-Jul 28, 1999:
-	added test in defn() to catch function foo(foo), which
-	otherwise recurses until core dump.  thanks to arnold
-	robbins for noticing this.
-
-Jun 20, 1999:
-	added *bp in gettok in lex.c; appears possible to exit function
-	without terminating the string.  thanks to russ cox.
-
-Jun 2, 1999:
-	added function stdinit() to run to initialize files[] array,
-	in case stdin, etc., are not constants; some compilers care.
-
-May 10, 1999:
-	replaced the ERROR ... FATAL, etc., macros with functions
-	based on vprintf, to avoid problems caused by overrunning
-	fixed-size errbuf array.  thanks to ralph corderoy for the
-	impetus, and for pointing out a string termination bug in
-	qstring as well.
-
-Apr 21, 1999:
-	fixed bug that caused occasional core dumps with commandline
-	variable with value ending in \.  (thanks to nelson beebe for
-	the test case.)
-
-Apr 16, 1999:
-	with code kindly provided by Bruce Lilly, awk now parses
-	/=/ and similar constructs more sensibly in more places.
-	Bruce also provided some helpful test cases.
-
-Apr 5, 1999:
-	changed true/false to True/False in run.c to make it
-	easier to compile with C++.  Added some casts on malloc
-	and realloc to be honest about casts; ditto.  changed
-	ltype int to long in struct rrow to reduce some 64-bit
-	complaints; other changes scattered throughout for the
-	same purpose.  thanks to Nelson Beebe for these portability
-	improvements.
-
-	removed some horrible pointer-int casting in b.c and elsewhere
-	by adding ptoi and itonp to localize the casts, which are
-	all benign.  fixed one incipient bug that showed up on sgi
-	in 64-bit mode.
-
-	reset lineno for new source file; include filename in error
-	message.  also fixed line number error in continuation lines.
-	(thanks to Nelson Beebe for both of these.)
-
-Mar 24, 1999:
-	Nelson Beebe notes that irix 5.3 yacc dies with a bogus
-	error; use a newer version or switch to bison, since sgi
-	is unlikely to fix it.
-
-Mar 5, 1999:
-	changed isnumber to is_number to avoid the problem caused by
-	versions of ctype.h that include the name isnumber.
-
-	distribution now includes a script for building on a Mac,
-	thanks to Dan Allen.
-
-Feb 20, 1999:
-	fixed memory leaks in run.c (call) and tran.c (setfval).
-	thanks to Stephen Nutt for finding these and providing the fixes.
-
-Jan 13, 1999:
-	replaced srand argument by (unsigned int) in run.c;
-	avoids problem on Mac and potentially on Unix & Windows.
-	thanks to Dan Allen.
-
-	added a few (int) casts to silence useless compiler warnings.
-	e.g., errorflag= in run.c jump().
-
-	added proctab.c to the bundle outout; one less thing
-	to have to compile out of the box.
-
-	added calls to _popen and _pclose to the win95 stub for
-	pipes (thanks to Steve Adams for this helpful suggestion).
-	seems to work, though properties are not well understood
-	by me, and it appears that under some circumstances the
-	pipe output is truncated.  Be careful.
-
-Oct 19, 1998:
-	fixed a couple of bugs in getrec: could fail to update $0
-	after a getline var; because inputFS wasn't initialized,
-	could split $0 on every character, a misleading diversion.
-
-	fixed caching bug in makedfa: LRU was actually removing
-	least often used.
-
-	thanks to ross ridge for finding these, and for providing
-	great bug reports.
-
-May 12, 1998:
-	fixed potential bug in readrec: might fail to update record
-	pointer after growing.  thanks to dan levy for spotting this
-	and suggesting the fix.
-
-Mar 12, 1998:
-	added -V to print version number and die.
-
-[notify dave kerns, dkerns@dacsoup.ih.lucent.com]
-
-Feb 11, 1998:
-	subtle silent bug in lex.c: if the program ended with a number
-	longer than 1 digit, part of the input would be pushed back and
-	parsed again because token buffer wasn't terminated right.
-	example:  awk 'length($0) > 10'.  blush.  at least i found it
-	myself.
-
-Aug 31, 1997:
-	s/adelete/awkdelete/: SGI uses this in malloc.h.
-	thanks to nelson beebe for pointing this one out.
-
-Aug 21, 1997:
-	fixed some bugs in sub and gsub when replacement includes \\.
-	this is a dark, horrible corner, but at least now i believe that
-	the behavior is the same as gawk and the intended posix standard.
-	thanks to arnold robbins for advice here.
-
-Aug 9, 1997:
-	somewhat regretfully, replaced the ancient lex-based lexical
-	analyzer with one written in C.  it's longer, generates less code,
-	and more portable; the old one depended too much on mysterious
-	properties of lex that were not preserved in other environments.
-	in theory these recognize the same language.
-
-	now using strtod to test whether a string is a number, instead of
-	the convoluted original function.  should be more portable and
-	reliable if strtod is implemented right.
-
-	removed now-pointless optimization in makefile that tries to avoid
-	recompilation when awkgram.y is changed but symbols are not.
-
-	removed most fixed-size arrays, though a handful remain, some
-	of which are unchecked.  you have been warned.
-
-Aug 4, 1997:
-	with some trepidation, replaced the ancient code that managed
-	fields and $0 in fixed-size arrays with arrays that grow on
-	demand.  there is still some tension between trying to make this
-	run fast and making it clean; not sure it's right yet.
-
-	the ill-conceived -mr and -mf arguments are now useful only
-	for debugging.  previous dynamic string code removed.
-
-	numerous other minor cleanups along the way.
-
-Jul 30, 1997:
-	using code provided by dan levy (to whom profuse thanks), replaced
-	fixed-size arrays and awkward kludges by a fairly uniform mechanism
-	to grow arrays as needed for printf, sub, gsub, etc.
-
-Jul 23, 1997:
-	falling off the end of a function returns "" and 0, not 0.
-	thanks to arnold robbins.
-
-Jun 17, 1997:
-	replaced several fixed-size arrays by dynamically-created ones
-	in run.c; added overflow tests to some previously unchecked cases.
-	getline, toupper, tolower.
-
-	getline code is still broken in that recursive calls may wind
-	up using the same space.  [fixed later]
-
-	increased RECSIZE to 8192 to push problems further over the horizon.
-
-	added \r to \n as input line separator for programs, not data.
-	damn CRLFs.
-
-	modified format() to permit explicit printf("%c", 0) to include
-	a null byte in output.  thanks to ken stailey for the fix.
-
-	added a "-safe" argument that disables file output (print >,
-	print >>), process creation (cmd|getline, print |, system), and
-	access to the environment (ENVIRON).  this is a first approximation
-	to a "safe" version of awk, but don't rely on it too much.  thanks
-	to joan feigenbaum and matt blaze for the inspiration long ago.
-
-Jul 8, 1996:
-	fixed long-standing bug in sub, gsub(/a/, "\\\\&"); thanks to
*** 5975 LINES SKIPPED ***