misc/135932: 'strtol' doesn't reset errno to 0 when converting MAX_INT=2147483647

[Yuri]

>Number:         135932
>Category:       misc
>Synopsis:       'strtol' doesn't reset errno to 0 when converting MAX_INT=2147483647
>Confidential:   no
>Severity:       non-critical
>Priority:       medium
>Responsible:    freebsd-bugs
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Mon Jun 22 20:20:01 UTC 2009
>Closed-Date:
>Last-Modified:
>Originator:     Yuri
>Release:        72-STABLE
>Organization:
n/a
>Environment:
>Description:
When strtol is supplied string 2147483647 it's impossible to distinguish between overflow and non-overflow situation since return value is the same one that flags overflow (MAX_INT) and strtol doesn't clear errno in this case.

strtol should set errno=0 in this case to avoid ambiguity.

errno can accidentally be set to 34 (ERANGE) before the call and this will affect the conversion decision in this case. And requiring all programs to reset errno before the call isn't right too since the proposed here simple fix fixes this ambiguity for every program.

-- testcase --
#include <stdlib.h>
#include <limits.h>
#include <errno.h>
#include <stdio.h>

main() {
  const char *nptr = "2147483647";
  char *endptr;
  errno=34; // ERANGE
  int res = ::strtol(nptr, &endptr, 10);
  printf("res=%i errno=%i\n", res, errno);
}

--- output --- 
res=2147483647 errno=34

>How-To-Repeat:

>Fix:


>Release-Note:
>Audit-Trail:
>Unformatted: