August OpenZFS Leadership Meeting

[Matthew Ahrens]

Thanks to everyone who participated at this month's meeting.  We discussed
OpenZFS DevSummit talks; the Linux Plumbers Conference; 32-bit kernel
atomic reads; zfs share -a on Linux; and dedup improvements.

The video is available here: https://www.youtube.com/watch?v=zl4CUMOIi18

Notes (thanks Karyn):

   -

   OpenZFS DevSummit talks (matt)
   -

      Lots of great talks so far, but there is still room.
      -

      If you forgot to submit a talk before yesterday’s deadline, please
      reach out right away. Matt will announce selected speakers in 2 weeks, so
      there is a small grace period.
      -

      Registration is open now, and Matt will publicize this more once we
      announce the speakers.
      -

   Linux Plumbers conf (Serapheim)
   -

      George WIlson and Serapheim will be at the Conference if you want to
      talk ZFS or Debugging the Linux Kernel
      -

      If so, please reach out so the ZFS folks can meet up and talk about
      all things ZFS.
      -

   OpenZFS on 32-bit architectures (relevant for FreeBSD, Linux, …): plain
   reads of 64-bit “atomics” (Andriy)
   -

      Issue discovered: We use atomic operations to modify 64-bit values
      that we treat as atomic, but to read them we use plain C assignment
      operations, which works ok for 64-bit platforms, but not for
32-bit. Those
      plain reads are implemented as 2 instructions there (32-bit), which is
      incorrect.
      -

      At this point it is mostly theoretical (its is rare to hit a write
      while crossing the 32-bit boundary) but it is still a valid concern.
      -

      The code paths are not very critical - mostly stats that we send to
      user space.
      -

      Proposal for discussion: Should we introduce some kind of format
      wrappers for those operations?
      -

         Paul Dagnelie: Has done some similar work for the SPL back in the
         day and this should be straightforward.
         -

            Question: How many 32-bit platforms do we actually care for at
            this point?
            -

         Andriy: Have heard of a few 32-bit Intel users out there
         -

         Tom Caputi: Has some experience for compile-time checks for
         structures of these nature.
         -

      It seems like people are for it. Andriy will first talk with FreeBSD
      developers, figure out how they want to proceed, and try to
propose designs
      common for all platforms.
      -

   zfs share -a (George)
   -

      Issue: Stale file handles using zfs share -a and the sharenfs
      property, by restarting or rebooting the nfs server.
      -

         The reason is that ZFS plugs into SystemD in a
         cumbersome/non-natural way.
         -

         NFS shares actually start before the ZFS service. Mountd responds
         to the client before the ZFS share is actually completed
leading to the
         stale handles.
         -

      Hacky Approach:
      -

         Introduce a new component to zfs share. Instead of zfs share -a ,
         zfs share -g (short for generate)
         -

         The idea is to add ZFS logic that ties it closer to NFS
         -

         Change of dependency ordering, zfs share -g generates the handles
         for NFS, before NFS, and NFS consumes those generated handles.
         -

         Cons: A bit hacky for now
         -

         Pros: Simple and fast
         -

         Plan: To do more redesign on top of this for a cleaner solution
         -

         Others that have similar issues with George/Delphix feel free to
         reach out.
         -

            Tom Caputi: Datto had issues with NFS in the past, and don't
            use sharenfs, but they are interested on having control over this
            functionality
            -

            Allan Jude: Seems like FreeBSD already does something similar,
            so there may be some convergence of goals.
            -

      Submitted this as lightning talk at the Dev Summit
      -

   Anyone interested in working on implementing Matt’s ideas for dedup
   ceiling and on disk format log (
   http://open-zfs.org/w/images/8/8d/ZFS_dedup.pdf) please contact
   allan at klarasystems.com to discuss. (Allan)
   -

      Problems using the feature for Terabytes of data due to known memory
      issues
      -

      There is budget for a contract to implement the limit of the size of
      the DDT proposed during one of the past Dev Summits
      -

      Feel free to reach out if you are interested in undertaking that work


On Mon, Aug 19, 2019 at 12:56 PM Matthew Ahrens <mahrens at delphix.com> wrote:

> The next OpenZFS Leadership meeting will be held tomorrow, August 20,
> 1pm-2pm Pacific time.
>
> Everyone is welcome to attend and participate, and we will try to keep the
> meeting on agenda and on time.  The meetings will be held online via Zoom,
> and recorded and posted to the website and YouTube after the meeting.
>
> The agenda for the meeting will be a discussion of the projects listed in
> the agenda doc.
>
> For more information and details on how to attend, as well as notes and
> video from the previous meeting, please see the agenda document:
>
>
> https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit
>
> --matt
>