[Bug 209842] backtrace(3) function can return negative

[bugzilla-noreply at freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=209842

            Bug ID: 209842
           Summary: backtrace(3) function can return negative
           Product: Base System
           Version: 10.3-RELEASE
          Hardware: arm
                OS: Any
            Status: New
          Severity: Affects Only Me
          Priority: ---
         Component: misc
          Assignee: freebsd-bugs at FreeBSD.org
          Reporter: phk at FreeBSD.org

The backtrace(3) function returns size_t which is unsigned:

     size_t
     backtrace(void **addrlist, size_t len);

And is documentet as returning:

   RETURN VALUES
     The backtrace() function returns the number of elements that were filled
     in the backtrace. [...]

The implementation in contrib/libexecinfo/unwind.c clearly knows that the
return value can be all-ones:

        if (ctx.n != (size_t)~0 && ctx.n > 0)
                ctx.arr[--ctx.n] = NULL;        /* Skip frame below __start */

        return ctx.n;

This happens on a BeagleBoneBlack running

    10.3-STABLE FreeBSD 10.3-STABLE #0 r300092

I am not sure what the proper fix is here, nor for that matter what the problem
might be that causes it to return the all-ones value in the first place.

But at the very least ctx.n should be slammed to zero before returning it, so
the code calling backtrace(3) doesn't run off the end of the world.

-- 
You are receiving this mail because:
You are the assignee for the bug.