Cleaning up FILE in stdio..

[Peter Jeremy]

On Wed, Feb 27, 2008 at 11:38:33AM -0500, John Baldwin wrote:
>> You could change _file from 'short' to 'unsigned short' without breaking
>> the ABI - this would allow either 65535 or 65536 file descriptors (I'm
>> not sure whether _file == -1 is special or not).  This would postpone
>> the problem for some time.
>
>-1 is used a lot in the stdio code for file's not backed by an fd.  My problem 
>though is that this doesn't help with existing binaries that are already 
>compiled (which is what I have to deal with).  Had fileno() not been inlined 
>I would have been ok, but that's pretty much done for me as far as my current 
>problem on 6.x.  Had I just been able to change FILE * and not had inlines, 
>then a new fopen would have worked fine in my case.

My suggestion was based on short and ushort having the same size and
(short)-1 and (ushort)65535 having the same bit pattern.  Any code
accessing fileno() should not be checking or validating the result
but just passing it to low-level I/O routines.  This would provide the
following:
               Existing code     New code
		   short          ushort
    FD         sign-extended   zero-extended
   -1              -1           65535
    0..32767        0..32767        0..32767
32768..65534   -32768..-2 [*]   32768..65534
 >65534          EMFILE            EMFILE

[*] This could potentially be fixed using libc or kernel shims.

>> e) Don asbestos underwear and re-arrange struct __sFILE to grow _file etc.
>
>We can't do e) because thanks to symbol versioning, 8.x and 9.x will have 
>libc.so.7, so a 7.0 binary will still use the brand new libc, so it has to 
>preserve the ABI of the currently exported fields pretty much forever.

Erk.  I forgot about that.

>think we can get away with renaming '_file' to '_ofile' and adding a new 'int 
>_file' at the bottom of the struct and making sure '_ofile' is always in sync 
>(when possible, truncated when _file is too bug).

Truncation opens up the possibility that old executables could fopen()
lots of files (without getting any indication of a problem) and then
use fileno() to reference a truncated _ofile - causing it to access
some totally unrelated file.  Admittedly, that is no worse than would
happen today.

>Also, I think we can do the new _file in HEAD for 8.0 w/o any worries.  I 
>don't think waiting until 9.0 buys anything there.

I was thinking in terms of changing _file to int without backward
compatibility.  I'm not sure that that could be done for 8.0 (though,
as you have pointed out, it can't be done at all).

>  Given that, I think I'd 
>rather just patch the current stable branches to handle the edge case better 
>and work on making _file an int in HEAD (with the ABI compat _ofile).

The EMFILE patch is definitely a good interim step and I support your
efforts in removing the limit on the number of open files.

-- 
Peter Jeremy
Please excuse any delays as the result of my ISP's inability to implement
an MTA that is either RFC2821-compliant or matches their claimed behaviour.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 187 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-arch/attachments/20080228/7b943bad/attachment.pgp