svn commit: r306782 - in head: contrib/netbsd-tests/lib/libc/locale usr.bin/localedef

[Baptiste Daroussin]

Author: bapt
Date: Thu Oct  6 19:46:43 2016
New Revision: 306782
URL: https://svnweb.freebsd.org/changeset/base/306782

Log:
  localedef: Fix ctype dump (fixed wide spread errors)
  
  This commit is from John Marino in dragonfly with the following commit log:
  
  ====
  This was a CTYPE encoding error involving consecutive points of the same
  ctype.  It was reported by myself to Illumos over a year ago but I was
  unsure if it was only happening on BSD.  Given the cause, the bug is also
  present on Illumos.
  
  Basically, if consecutive points were of the exact same ctype, they would
  be defined as a range regardless.  For example, all of these would be
  considered equivalent:
  
    <A> ... <C>, <H>  (converts to <A> .. <H>)
    <A>, <B>, <H>     (converts to <A> .. <H>)
    <A>, <J> ... <H>  (converts to <A> .. <H>)
  
  So all the points that shouldn't have been defined got "bridged" by the
  extreme points.
  
  The effects were recently reported to FreeBSD on PR 213013.  There are
  countless places were the ctype flags are misdefined, so this is a major
  fix that has to be MFC'd.
  ====
  
  This reveals a bad change I did on the testsuite: while 0x07FF is a valid
  unicode it is not used yet (reserved for future use)
  
  PR:		213013
  Submitted by:	marino@
  Reported by:	Kurtis Rader <krader at skepticism.us>
  Obtained from:	Dragonfly
  MFC after:	1 month

Modified:
  head/contrib/netbsd-tests/lib/libc/locale/t_mbstowcs.c
  head/usr.bin/localedef/ctype.c

Modified: head/contrib/netbsd-tests/lib/libc/locale/t_mbstowcs.c
==============================================================================
--- head/contrib/netbsd-tests/lib/libc/locale/t_mbstowcs.c	Thu Oct  6 19:41:09 2016	(r306781)
+++ head/contrib/netbsd-tests/lib/libc/locale/t_mbstowcs.c	Thu Oct  6 19:46:43 2016	(r306782)
@@ -88,7 +88,7 @@ static struct test {
 		0xFFFF, 0x5D, 0x5B, 0x10000, 0x10FFFF, 0x5D, 0x0A
 	},
 #ifdef __FreeBSD__
-	{	 1, -1, -1,  1,  1, -1, 1,  1,  1, 1, -1,  1,  1, -1, -1,
+	{	 1, -1, -1,  1,  1, -1, -1,  1,  1, 1, -1,  1,  1, -1, -1,
 #else
 	{	 1, -1, -1,  1,  1, -1, -1,  1,  1, -1, -1,  1,  1, -1, -1,
 #endif

Modified: head/usr.bin/localedef/ctype.c
==============================================================================
--- head/usr.bin/localedef/ctype.c	Thu Oct  6 19:41:09 2016	(r306781)
+++ head/usr.bin/localedef/ctype.c	Thu Oct  6 19:46:43 2016	(r306782)
@@ -407,9 +407,9 @@ dump_ctype(void)
 			continue;
 		}
 
-		if ((last_ct != NULL) && (last_ct->ctype == ctn->ctype)) {
+		if ((last_ct != NULL) && (last_ct->ctype == ctn->ctype) &&
+		    (last_ct->wc + 1 == wc)) {
 			ct[rl.runetype_ext_nranges-1].max = wc;
-			last_ct = ctn;
 		} else {
 			rl.runetype_ext_nranges++;
 			ct = realloc(ct,
@@ -417,8 +417,8 @@ dump_ctype(void)
 			ct[rl.runetype_ext_nranges - 1].min = wc;
 			ct[rl.runetype_ext_nranges - 1].max = wc;
 			ct[rl.runetype_ext_nranges - 1].map = ctn->ctype;
-			last_ct = ctn;
 		}
+		last_ct = ctn;
 		if (ctn->tolower == 0) {
 			last_lo = NULL;
 		} else if ((last_lo != NULL) &&