svn commit: r352795 - head/lib/libc/sys

[Bruce Evans]

On Tue, 1 Oct 2019, Ed Maste wrote:

> On Tue, 1 Oct 2019 at 12:23, Brooks Davis <brooks at freebsd.org> wrote:
>>
>> This isn't true with CHERI and as a result I've moved the variadic
>> argument handling (except for syscall() and __syscall()) into libc.
>
> My grep found: open, openat, fcntl, semsys, msgsys, shmsys
> Is that the full list?

I already wrote that this is quite broken for open and fcntl in POSIX.
hecking some details shows that it is more fundamentally broken than I
thought:
- for open(), the type of the mode argument passed by the caller is
   unspecified.  Whatever it is, it is "taken" as type mode_t, whatevr
   "taking" is.  Since historical mode_t has only 16 bits, it can be
   represented by int even on systems with 16-bit ints, so the caller
   can start with either mode_t or int provided mode_t is no larger than
   historical mode_t and ints are either larger than 16 bits or 16 bits
   and not too exotic (the sign bit might cause problems if not 2's
   complement)
- for fcntl() with F_SETOWN, the type of the pid argument passed by the
   caller is unspecified.  Whatever it is, it is "taken" as type int.
   Thus if pid_t is larger than int, passing all possible values of
   pid_t is impossible.  If also PID_MAX <= INT_MAX and all values of
   pid_t are actually <= PID_MAX, then all possible (positive) values
   can be passed, but the iplementation may have to do extra work to
   properly break a passed __default_promotion_of(pid_t) type by "taking"
   it as an int.

   This was discussed on the POSIX list recently.  IMO it is too late and
   not useful to change the old specification to "take" the arg as anything
   except int.  So pid_t might as well be specified as being a signed integer
   type whose default promotion is int.  It is currently specified as being
   a signed integer type (with any size or exoticness).

Bruce