ports/74537: editors/vim issue: report invalid size of multi-byte character

[Wataru Gotoh]

>Number:         74537
>Category:       ports
>Synopsis:       editors/vim issue: report invalid size of multi-byte character
>Confidential:   no
>Severity:       non-critical
>Priority:       medium
>Responsible:    freebsd-ports-bugs
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Mon Nov 29 21:20:01 GMT 2004
>Closed-Date:
>Last-Modified:
>Originator:     Wataru Gotoh
>Release:        FreeBSD 5.3-RELEASE-p1 i386
>Organization:
>Environment:
System: GENERIC kernel.

>Description:
editors/vim on FreeBSD 5-stable (include 5.3R) report invalid size of multi-byte character.
this issue poses some problem, on display text, on moving cursor, and more.
vim call "mblen()" for make "mb_bytelen_tab" table at vim63/src/mbyte.c in function named mb_init().
current spec of mblen() is not suppose for make this table.

>How-To-Repeat:
test code of make table.

#include <stdio.h>
#include <stdlib.h>
#include <locale.h>

int main(void)
{
   int i, n;
   char buf[2];

   fprintf(stdout, "%s\n", setlocale(LC_ALL, "ja_JP.eucJP"));
   
   for (i = 0; i < 256; i++) {
      buf[0] = i;
      buf[1] = '\0';
      if (i == '\0') {
         n = 1;
      } else {
         if (mblen(buf, (size_t)1) <= 0) {
            n = 2;
         } else {
            n = 1;
         }
      }
      fprintf(stdout, "%d,", n);

      if ((i + 1) % 32 == 0) {
         fprintf(stdout, "\n");
      }
   }

   return 0;
}

valid case: (on 5.2.1R)
ja_JP.eucJP
1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,
1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,
1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,
1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,
2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,
2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,
2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,
2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,

invalid case: (on 5.3R)
ja_JP.eucJP
1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,
1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,
1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,
1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,
2,1,2,1,2,1,2,1,2,1,2,1,2,1,2,1,2,1,2,1,2,1,2,1,2,1,2,1,2,1,2,1,
2,1,2,1,2,1,2,1,2,1,2,1,2,1,2,1,2,1,2,1,2,1,2,1,2,1,2,1,2,1,2,1,
2,1,2,1,2,1,2,1,2,1,2,1,2,1,2,1,2,1,2,1,2,1,2,1,2,1,2,1,2,1,2,1,
2,1,2,1,2,1,2,1,2,1,2,1,2,1,2,1,2,1,2,1,2,1,2,1,2,1,2,1,2,1,2,1,

>Fix:
clear status of mblen(), before factual calling.

patch for editors/vim

diff -urN vim.orig/files/patch-mbyte_c vim/files/patch-mbyte_c
--- vim.orig/files/patch-mbyte_c	Thu Jan  1 09:00:00 1970
+++ vim/files/patch-mbyte_c	Tue Nov 30 04:16:47 2004
@@ -0,0 +1,10 @@
+--- mbyte.c.orig	Sun Nov 28 20:40:35 2004
++++ mbyte.c	Sun Nov 28 20:42:49 2004
+@@ -650,6 +650,7 @@
+ 		     * where mblen() returns 0 for invalid character.
+ 		     * Therefore, following condition includes 0.
+ 		     */
++		    mblen(NULL, 0);
+ 		    if (mblen(buf, (size_t)1) <= 0)
+ 			n = 2;
+ 		    else

>Release-Note:
>Audit-Trail:
>Unformatted: