FD_SETSIZE: signed or unsigned?

Bruce Evans brde at optusnet.com.au
Tue Jul 3 17:17:52 UTC 2012

On Mon, 2 Jul 2012, Benjamin Kaduk wrote:

> We've had an unsigned FD_SETSIZE since 2002 (r90135, by markm):
> %%%%%%%%%%%%%%%%%%%%%%%%
> Revision 90135 - (show annotations)
> Sun Feb 3 11:36:59 2002 UTC (10 years, 5 months ago) by markm
> File MIME type: text/plain
> File size: 7279 byte(s)
> Zero functional difference; make some integer constants unsigned, as
> they are used in unsigned context. This shuts lint(1) up in a few
> significant ways with "signed/unsigned" arithmetic warnings.
> %%%%%%%%%%%%%%%%%%%%%%%%
> Yet NetBSD, Linux, and OpenBSD all have signed (plain int) FD_SETSIZEs.
> This has led to various pieces of software introducing casts and workarounds 
> for the FreeBSD behavior, which (apparently) was itself made in order to 
> attempt to reduce the occurence of warnings.

I didn't like it at the time, and got the similar changes to NBBY and
howmany() backed out.  Only the unsigned constant in howmany() was
obviously wrong.  It caused promotion of nearby int args to unsigned,
so for example howmany(-1, 1) was UINT_MAX with type unsigned instead
of -1 with type int.  OTOH, howmany(-1L, 1) was -1L on arches with
sizeof(u_int) < sizeof(long) (assume no padding bits throughout), but
was (long)UINT_MAX on arches with sizeof(u_int) == sizeof(long).
(long)UINT_MAX is implementation-defined and is normally also -1L.

Maybe negative howmany()'s aren't useful, but the unsigned constant
also makes the type of howmany() unsigned when it should be int, giving
the same problem as for FD_SETSIZE.  You don't want to know about these
complications, and should let the default binary promotions work normally
by not having any unusual types hidden in the implementation of howmany().
Especially when previous and other implementations dont't have them.  If
you really want unsigned behaviour in howmany(), then you can pass it
unsigned args (howmany((x), 0U + (y)) would give the same behaviour as
the hidden 1U).

> Before I go and introduce such workarounds into our codebase for $work, I 
> wanted to check with the standards gurus to see whether there is any chance 
> that FreeBSD is what needs fixing, first.

I would prefer to change it back to plain int.

> Bruce touched on some related issues on freebsd-security a few months later 
> in 2002 ("[provos at citi.umich.edu: OpenBSD Security Advisory: Select Boundary 
> Condition]"), but I can't really derive anything decisive from it.

I had forgotten about that (but complained about magic casts to `unsigned int'
in file descriptor code recently :-)).

> In any case, it is clear that a negative value for FD_SETSIZE is nonsensical.

But comparing "int fd" with FD_SETSIZE makes sense, and having FD_SIZE a
different type than int mainly gives sign extension problems.  The
problems are mostly only potential, but the compiler doesn't know that
and gives actual problems by warning about the potential ones; -Werror
then makes these fatal.

IMO, unsigned types should never be used just because the values in them are
expected to be always nonnegative.  Use of unsigned types mainly gives
(un)sign extension bugs when unsigned types are mixed with signed types
in expressions.

> Doing a quick survey, it seems to mostly be used as the first argument to 
> select(2) (which is an int type argument), and as a bound check on fd numbers 
> (before using them in select).  Given the former, it seems dubious to 
> consider ever using a value between INT_MAX and UINT_MAX, which does leave 
> open what the actual type should be.

The value of (INT_MAX + 1U) actually makes sense: if you have fds numbered
0 to INT_MAX, then the number of fds is (INT_MAX + 1U).  The POSIX history
reminded me of this -- old versions had an off-by-1 error for nfds, with
nfds >= FD_SETSIZE an error, but it is nfds > FD_SETSIZE that is the error.
Anyway, nfds is int, so it cannot be (INT_MAX + 1U), and FD_SETSIZE being
(INT_MAX + 1U) is not useful since apart from being more than large
enough for anyone (at least with 32 bit ints), nfds can never equal it.

> POSIX is delightfully vague:
> %%%%%%%%%%%%%%%%%%%%%%
> The following shall be defined as a macro:
>    Maximum number of file descriptors in an fd_set structure.
> %%%%%%%%%%%%%%%%%%%%%%

The 2001 and 2007 (draft) versions are equally vague.


More information about the freebsd-standards mailing list