Re: Renaming O_NAMEDATTR to O_NAMEDSTREAMS Re: Size of "alternate data streams"/"resource forks" / O_NAMEDATTR Re: FreeBSD Status Report - Second Quarter 2025

On Wed, 3 Sep 2025 18:34:44 +0200
Cedric Blancher <cedric.blancher@gmail.com> wrote:

> On Tue, 2 Sept 2025 at 23:12, Rick Macklem <rick.macklem@gmail.com> wrote:
> >
> > On Tue, Sep 2, 2025 at 11:29 AM Dan Shelton <dan.f.shelton@gmail.com>
> > wrote:  
> > >
> > > On Sat, 30 Aug 2025 at 12:54, Lorenzo Salvadore <salvadore@freebsd.org>
> > > wrote:  
> > > > The top level system call interface is open(2)/openat(2) with the new
> > > > O_NAMEDATTR flag (called O_XATTR on Solaris).
> > > >
> > > > Most of the work has been committed to FreeBSD’s main for FreeBSD 15.
> > > > Once the ZFS patch makes it through review and gets pulled into
> > > > OpenZFS, the ZFS and NFSv4 support should work. There are also a
> > > > couple of manual pages currently under review in phabricator.
> > > >
> > > > The main thing left to do is update libarchive/tar so that large
> > > > extended attributes can be archived/retrieved. (The current FreeBSD
> > > > extended attribute mechanism is supported by libarchive, but will have
> > > > size constraints.)  
> > >
> > > 1. There are no size constraints. The main problem is different:
> > > SUN's original name is both a misnomer, and was hijacked by Linux to
> > > stuff in their broken clone of Windows extended attributes.
> > >
> > > The O_NAMEDATTR/O_XATTR files are in fact "alternate data streams",
> > > like on Windows, macOS (which calls them resource forks), and
> > > mainframe operating systems.
> > > They can have unlimited size (like the "main" data stream), and can
> > > even be sparse (SEEK_HOLE&friends). These are very different beats
> > > from "extended attributes".
> > >
> > > Background:
> > > Early versions of Windows started with EA (extended attributes), and
> > > then Windows NT4 adopted alternate data streams as a superior super
> > > set of the EA. Windows just couldn't get rid of the EAs, so they (and
> > > their evil Linux xattr clone) and their limitations keep haunting us.
> > >
> > > Funny is, EAs in Windows are nowadays just emulated via alternate data
> > > streams (stream "$EA" is the index, stream "$EA_INFORMATION" has the
> > > raw data).
> > >
> > > 2. Look at Solaris tar, which can handle unlimited size of O_XATTR
> > > streams  
> > It might be possible to put Solaris tar (from OpenSolaris) in ports.
> > I'll put it on my (currently rather long) todo list, unless someone
> > else is inspired
> > to look at it?
> >
> > As for libarchive, there are two problems:
> > - The way it is structured, it generates a linked list of a file's
> > extended attributes.
> >   As such, it is possible for very large ones to run into malloc()
> > failures (aka ENOMEM).
> >   To fix this would be a major re-write of libarchive, so I do not see
> > that happenning.
> > - The other is that it currently uses
> > extattr_get_file/extattr_set_file, which copies the
> >   entire attribute in one syscall. This actually works for ZFS
> > locally, but will not work
> >   for NFSv4.2 because there is a limit (currently a little over
> > 1Mbyte) on RPC message
> >   size.  
> >   --> This should be fixable via a patch that replaces the above  
> > syscalls with loops
> >        on read/write for file systems that support named attributes.
> > This is already on my todo list.  
> 
> I think the problem here is that people think O_XATTR/O_NAMEDATTRS are
> "attributes". Which implies a short amount of data, which is
> ABSOLUTELY NOT THE CASE!!
> 
> Maybe O_NAMEDATTR and O_XATTR should be renamed to O_NAMEDSTREAMS
> (plural!), as a clear and present warning that this stuff can store
> lots of data?

Would be good alias.

> For tar I really had to ask. Roland Mainz ex-SUN/ex-RH replied:
> ... O_XATTR (alias ADS support) is not stored as a string blob in
> Solaris tar+pax.
> tar(1) creates normal tar entries for a file, all with the filename
> "/dev/null" for backwards compatibility (so older SUN tar and tars on
> other platforms without ADS support just dump such data to /dev/null),
> and the real name of the ADS stream is in an extension header ...
> 
> (ADS means Alternate Data Stream)
> 
> Ced

-- 
WBR, @nuclight