du(1)/fts(3) integer overflow

[Peter Jeremy]

On Mon, 2004-Dec-13 11:42:17 -0000, henry wrote:
>I have noticed a problem with the fts(3) library or the way du(1) interacts
>with it.
>
>A 3.2TiB file gives the following output:
>> du -cs /fs/file
>3408720016      /fs/file
>-886247279      total
>
>This is because while stat(2) reports blocks as a 64bit number du(1) uses
>the 32bit value FTSENT.fts_number to store the result:

I think the large filesystem support is insufficiently exercised and
there will probably be more of these sort of things lurking around.

>The simplest change appears to be to make fts_number 64bit however this
>changes the fts(3) abi, so I am not sure if this is acceptable.

For 6.x, the ABI isn't fixed so fts_number can be changed to int64_t.
4.x doesn't really support large filesystems due to integer overflow
issues in UFS/FFS code so it's not really a problem there.

The 5.x ABI is fixed so there's no simple solution there.  Possible hacks
would be:
- Add a new 'fts_number64' at the end of FTSENT.  Since FTSENT is always
  managed by fts(3) and the documentation allows for undocumented fields,
  this should be permitted, though a "new" du(1) with an "old" libc
  would break badly.
- Move fts_number to the end of FTSENT and leave a 'long' hole where the
  existing fts_number is.  This changes the ABI but old programs remain
  compatible with the new fts.  (Though new programs break with the
  old fts).
- <Severe_kludge_alert>Have du(1) treat fts_pointer as an integer that
  it can concatenate to fts_number on 32-bit architectures:
    int64_t x = (int64_t)(ulong)p->fts_parent->fts_number |
	((int64_t)(ulong)p->fts_parent->fts_pointer) << 32;
    x += p->fts_statp->st_blocks;
    p->fts_parent->fts_number = (long)x;
    p->fts_parent->fts_pointer = (void *)(long)(x >> 32);
  etc. </Severe_kludge_alert>

-- 
Peter Jeremy