Renaming O_NAMEDATTR to O_NAMEDSTREAMS Re: Size of "alternate data streams"/"resource forks" / O_NAMEDATTR Re: FreeBSD Status Report - Second Quarter 2025

On Tue, 2 Sept 2025 at 23:12, Rick Macklem <rick.macklem@gmail.com> wrote:
>
> On Tue, Sep 2, 2025 at 11:29 AM Dan Shelton <dan.f.shelton@gmail.com> wrote:
> >
> > On Sat, 30 Aug 2025 at 12:54, Lorenzo Salvadore <salvadore@freebsd.org> wrote:
> > > The top level system call interface is open(2)/openat(2) with the new
> > > O_NAMEDATTR flag (called O_XATTR on Solaris).
> > >
> > > Most of the work has been committed to FreeBSD’s main for FreeBSD 15. Once the
> > > ZFS patch makes it through review and gets pulled into OpenZFS, the ZFS and
> > > NFSv4 support should work. There are also a couple of manual pages currently
> > > under review in phabricator.
> > >
> > > The main thing left to do is update libarchive/tar so that large extended
> > > attributes can be archived/retrieved. (The current FreeBSD extended attribute
> > > mechanism is supported by libarchive, but will have size constraints.)
> >
> > 1. There are no size constraints. The main problem is different:
> > SUN's original name is both a misnomer, and was hijacked by Linux to
> > stuff in their broken clone of Windows extended attributes.
> >
> > The O_NAMEDATTR/O_XATTR files are in fact "alternate data streams",
> > like on Windows, macOS (which calls them resource forks), and
> > mainframe operating systems.
> > They can have unlimited size (like the "main" data stream), and can
> > even be sparse (SEEK_HOLE&friends). These are very different beats
> > from "extended attributes".
> >
> > Background:
> > Early versions of Windows started with EA (extended attributes), and
> > then Windows NT4 adopted alternate data streams as a superior super
> > set of the EA. Windows just couldn't get rid of the EAs, so they (and
> > their evil Linux xattr clone) and their limitations keep haunting us.
> >
> > Funny is, EAs in Windows are nowadays just emulated via alternate data
> > streams (stream "$EA" is the index, stream "$EA_INFORMATION" has the
> > raw data).
> >
> > 2. Look at Solaris tar, which can handle unlimited size of O_XATTR streams
> It might be possible to put Solaris tar (from OpenSolaris) in ports.
> I'll put it on my (currently rather long) todo list, unless someone
> else is inspired
> to look at it?
>
> As for libarchive, there are two problems:
> - The way it is structured, it generates a linked list of a file's
> extended attributes.
>   As such, it is possible for very large ones to run into malloc()
> failures (aka ENOMEM).
>   To fix this would be a major re-write of libarchive, so I do not see
> that happenning.
> - The other is that it currently uses
> extattr_get_file/extattr_set_file, which copies the
>   entire attribute in one syscall. This actually works for ZFS
> locally, but will not work
>   for NFSv4.2 because there is a limit (currently a little over
> 1Mbyte) on RPC message
>   size.
>   --> This should be fixable via a patch that replaces the above
> syscalls with loops
>        on read/write for file systems that support named attributes.
> This is already on my todo list.

I think the problem here is that people think O_XATTR/O_NAMEDATTRS are
"attributes". Which implies a short amount of data, which is
ABSOLUTELY NOT THE CASE!!

Maybe O_NAMEDATTR and O_XATTR should be renamed to O_NAMEDSTREAMS
(plural!), as a clear and present warning that this stuff can store
lots of data?

For tar I really had to ask. Roland Mainz ex-SUN/ex-RH replied:
... O_XATTR (alias ADS support) is not stored as a string blob in
Solaris tar+pax.
tar(1) creates normal tar entries for a file, all with the filename
"/dev/null" for backwards compatibility (so older SUN tar and tars on
other platforms without ADS support just dump such data to /dev/null),
and the real name of the ADS stream is in an extension header ...

(ADS means Alternate Data Stream)

Ced
-- 
Cedric Blancher <cedric.blancher@gmail.com>
[https://plus.google.com/u/0/+CedricBlancher/]
Institute Pasteur