patch that makes d_fileno 64bits

[Gleb Kurtsou]

On (28/12/2014 23:02), Kirk McKusick wrote:
> > Date: Sat, 27 Dec 2014 18:32:48 -0500 (EST)
> > From: Rick Macklem <rmacklem at uoguelph.ca>
> > To: FreeBSD Filesystems <freebsd-fs at freebsd.org>,
> >         Kirk McKusick <mckusick at mckusick.com>, Gleb Kurtsou <gleb at freebsd.org>,
> >         Konstantin Belousov <kib at freebsd.org>
> > Subject: patch that makes d_fileno 64bits
> > 
> > Hi,
> > 
> > Kirk and Gleb Kurtsou (plus some others) are working through
> > the difficult job of changing ino_t to 64bits. (Changes to syscalls,
> > libraries, etc.)
> > 
> > This patch:
> > http://people.freebsd.org/~rmacklem/64bitfileno.patch
> > 
> > is somewhat tangential to the above, in that it changes the
> > d_fileno field of "struct dirent" and va_fileid to uint64_t.
> > It also includes adding a field called d_cookie to "struct dirent",
> > which is the position of the next directory entry in the underlying
> > file system. A majority of this patch are changes to the NFS code,
> > but it includes a simple "new struct dirent"->"old struct dirent32"
> > copy routine for getdirentries(2) and small changes to all the
> > file systems so they fill in the "new struct dirent".
> > 
> > This patch can be applied to head/current and the resultant kernel
> > should work fine, although I've only been able to test some of the
> > file systems. However, DO NOT propagate the changes to sys/sys/dirent.h
> > out to userland (/usr/include/sys/dirent.h) and build a userland from
> > it or things will get badly broken.
> > 
> > I don't know if Kirk and/or Gleb will find some of this useful for
> > their updates to project/ino64, but it will allow people to test
> > these changes. (It modifies the NFS server so that it no longer uses
> > the "cookie" args to VOP_READDIR(), but that part can easily
> > be removed from the patch.)
> > 
> > If folks can test this patch, I think it would be helpful for
> > the effort of changing ino_t to 64bits.
> > 
> > Have fun with it, rick
> 
> Thanks Rick, this does look useful. Since Gleb is leading the charge
> with changing ino_t to 64 bits, I will let him have final say on when
> it would be helpful to have this go into HEAD. But it does seem that
> it should be possible to do it before the other changes and independently
> of them since it only chnges the internal kernel interfaces. But perhaps
> I am missing something that Gleb or kib can point out.

Full 64-bit ino_t patch is now committed to projects/ino64. Reviews are
welcome and encouraged.

It would be interesting to compare Rick's changes to VOP_READDIR
implementations to the code I had in 2011. As far as I remember I didn't
get to the point of using d_off in NFS. I intend to get back to it after
we are done with ino64.

Preferably those changes should be committed to CURRENT in several
chunks with at least a couple of weeks intervals in between, otherwise
IMHO that would be too much of a change for the interfaces virtually
every program uses:
1. Merge projects/ino64
2. Commit VOP_READDIR changes to populate d_off / d_cookie
3. Use d_off in NFS

On projects/ino64 branch:
- No problems found so far with kernel compat shims. I'm running tests
  with new kernel and old userland, mostly things like make universe,
  /usr/tests, pjdfstest, blogbench, dbench.
- I had another look at changing mode_t to 32-bit. At this point I would
  suggest to keep mode_t as 16-bit. Compat shims won't be trivial to
  implement and test mostly because of the usage in ipc subsystems:
  struct ipc_perm, ksem, shmfd. kinfo_file and libprocstat are already
  extended to use 32-bit for mode_t equivalent, so we may keep them as
  is.

My current TODO:
- Double check ABI with shlib-compat (including lib32)
- Bump majors for shlibs without symbol versioning. It is my
  understanding that we should bump them all. kib? ABI changes for at
  least libutil, libufs and libarchive
- Retest with new libc.so only
- Install complete ino64 userland and retest
- Ask for help with running complete ports build


> It seems to me that the cookies calculation could be taken out of the
> VOP_GETDIRENTRIES interface since NFS is the only client of it.

Do you mean cookies and ncookies arguments? That would be nice. And
while there replace uio with handler function argument for dirent
processing (was mentioned elsewhere). E.g. NFS doesn't need entries to
be stored in the buffer and can process them directly.