kern/148581: [libc] fopen(3) fails with EMFILE if there are more than SHORT_MAX fds open

[Jilles Tjoelker]

The following reply was made to PR kern/148581; it has been noted by GNATS.

From: Jilles Tjoelker <jilles at stack.nl>
To: bug-followup at FreeBSD.org, manishv at lineratesystems.com
Cc:  
Subject: Re: kern/148581: [libc] fopen(3) fails with EMFILE if there are
 more than SHORT_MAX fds open
Date: Sat, 17 Jul 2010 14:06:00 +0200

 Strictly speaking, your very dirty supposedly safe fix breaks binary
 compatibility because fileno() (in non-threaded programs only) and
 fileno_unlocked() are macros that hard-code the location and size of the
 _file field into binaries. If you have code compiled before the change
 in the same process as code compiled after the change, it might happen
 that data is read/written from/to the wrong descriptor.
 
 What may work is extending FILE (although I'm not entirely sure that
 there is noone that allocates their own FILE) with a 32-bit file
 descriptor field. If the file descriptor exceeds 32767, the 16-bit field
 then contains -1 and fileno() in old binaries will return that. This
 will at least fail safely although fileno() is not defined to return
 error conditions (but it has always returned -1 if the FILE is not
 associated with a file descriptor).
 
 -- 
 Jilles Tjoelker