Re: Renaming O_NAMEDATTR to O_NAMEDSTREAMS Re: Size of "alternate data streams"/"resource forks" / O_NAMEDATTR Re: FreeBSD Status Report - Second Quarter 2025

On Thu, Oct 2, 2025 at 4:55 AM Aurélien Couderc
<aurelien.couderc2002@gmail.com> wrote:
>
> On Sat, Sep 13, 2025 at 9:05 PM Vadim Goncharov <vadimnuclight@gmail.com> wrote:
> >
> > On Wed, 10 Sep 2025 15:06:47 -0700
> > Rick Macklem <rick.macklem@gmail.com> wrote:
> >
> > > > > > > > > 1. There are no size constraints. The main problem is different:
> > > > > > > > > SUN's original name is both a misnomer, and was hijacked by
> > > > > > > > > Linux to stuff in their broken clone of Windows extended
> > > > > > > > > attributes.
> > > > > > > > >
> > > > > > > > > The O_NAMEDATTR/O_XATTR files are in fact "alternate data
> > > > > > > > > streams", like on Windows, macOS (which calls them resource
> > > > > > > > > forks), and mainframe operating systems.
> > > > > > > > > They can have unlimited size (like the "main" data stream), and
> > > > > > > > > can even be sparse (SEEK_HOLE&friends). These are very different
> > > > > > > > > beats from "extended attributes".
> > > > > > > > >
> > > > > > > > > Background:
> > > > > > > > > Early versions of Windows started with EA (extended attributes),
> > > > > > > > > and then Windows NT4 adopted alternate data streams as a
> > > > > > > > > superior super set of the EA. Windows just couldn't get rid of
> > > > > > > > > the EAs, so they (and their evil Linux xattr clone) and their
> > > > > > > > > limitations keep haunting us.
> > > > > > > > >
> > > > > > > > > Funny is, EAs in Windows are nowadays just emulated via
> > > > > > > > > alternate data streams (stream "$EA" is the index, stream
> > > > > > > > > "$EA_INFORMATION" has the raw data).
> > > > > > > > >
> > > > > > > > > 2. Look at Solaris tar, which can handle unlimited size of
> > > > > > > > > O_XATTR streams
> > > > > > > > It might be possible to put Solaris tar (from OpenSolaris) in
> > > > > > > > ports. I'll put it on my (currently rather long) todo list, unless
> > > > > > > > someone else is inspired
> > > > > > > > to look at it?
> > > > > > > >
> > > > > > > > As for libarchive, there are two problems:
> > > > > > > > - The way it is structured, it generates a linked list of a file's
> > > > > > > > extended attributes.
> > > > > > > >   As such, it is possible for very large ones to run into malloc()
> > > > > > > > failures (aka ENOMEM).
> > > > > > > >   To fix this would be a major re-write of libarchive, so I do not
> > > > > > > > see that happenning.
> > > > > > > > - The other is that it currently uses
> > > > > > > > extattr_get_file/extattr_set_file, which copies the
> > > > > > > >   entire attribute in one syscall. This actually works for ZFS
> > > > > > > > locally, but will not work
> > > > > > > >   for NFSv4.2 because there is a limit (currently a little over
> > > > > > > > 1Mbyte) on RPC message
> > > > > > > >   size.
> > > > > > > >   --> This should be fixable via a patch that replaces the above
> > > > > > > > syscalls with loops
> > > > > > > >        on read/write for file systems that support named
> > > > > > > > attributes. This is already on my todo list.
> > > > > > >
> > > > > > > I think the problem here is that people think O_XATTR/O_NAMEDATTRS
> > > > > > > are "attributes". Which implies a short amount of data, which is
> > > > > > > ABSOLUTELY NOT THE CASE!!
> > > > > > >
> > > > > > > Maybe O_NAMEDATTR and O_XATTR should be renamed to O_NAMEDSTREAMS
> > > > > > > (plural!), as a clear and present warning that this stuff can store
> > > > > > > lots of data?
> > > > > >
> > > > > > I like the idea, because it fixes the problem that the
> > > > > > |O_ATTR|/|O_NAMEDATTR| are treated like attributes, while SUN's
> > > > > > intention was to implement "Alternate Data Streams" (which is a
> > > > > > superset of "extended attributes" as seen in Win32 EA and Linux XATTR,
> > > > > > but with none of their limitations and problems) for Windows, MacOS,
> > > > > > VMS and mainframe OS support.
> > > > > >
> > > > > > Re Plural: NO, please read openat(2): You open ONE named (attribute)
> > > > > > stream by passing this flag and the name, or the index (dir) by
> > > > > > passing "." as name. One attribute == Singular.
> > > > >
> > > > > Please also update the related #defines:
> > > > > 1. for openat()
> > > > > O_NAMEDSTREAM
> > > > >
> > > > > 2. for pathconf()
> > > > > _PC_NAMEDSTREAM_ENABLED
> > > > > _PC_NAMEDSTREAM_EXISTS
> > > > >
> > > > > That is actually a naming convention better than XATTR.
> > > Well, I've been trying to ignore this, but I guess I have to respond...
> > > (Here's some rather random comments.)
> > > - The bike shed is already painted green and the paint is dry.
> > >   I do not see any reason to repaint it blue.
> > > - As others have noted, every vendor has used a different name
> > >   for this mechanism.
> > >   The only place I am aware of where a standards document
> > >   describes them is the NFSv4 RFCs and these documents
> > >   call them "named attribute".
> > > - I'm not a Windows guy and when I read "stream" I think of
> > >   the SVR4 mechanism for handling network connections,
> > >   which makes me shy of "namedstream".
> >
> > Fortunately, SVR4 were dead already in previous century. We may safely
> > forget them. While being non-Windows guy, the NTFS is very good designed and
> > deserves knowledge about it's format at least in tutorial level (ReiserFS
> > tried to achieve same flexibility). They still call everything pertaining to
> > file as "attributes", though - so those are just several $DATA attributes (in
> > addition to other attributes, such as ACLs), default one (vast majority of
> > regular files) just has empty name.
> >
> > > - I chose a name other than "extended attribute" (which is
> > >   what Solaris calls it) because FreeBSD already uses
> > >   extended attribute for another mechanism (the one that
> > >   I think originated in SGI Irix and is in Linux).
> >
> > Yes, in NTFS attribute $EA could be one of them.
> >
> > > - If you think Oracle is going to change O_XATTR to some
> > >   other name, well I'd be really surprised if they did.
> > >   (I thought Solaris was on life support at this time.)
> > >
> > > FreeBSD is now in "slush" for the FreeBSD15 release,
> > > which means commits are supposed to be focused on
> > > bugfixes and I do not see renaming these as bugfixes.
> > >
> > > If I see a groundswell of support for the name change
> > > by what I will refer to as FreeBSD users that post
> > > comments regularly, I will consider doing it after
> > > the FreeBSD15 release goes out.
> > > (Sorry, but from what I have seen, the list of people posting
> > > to this thread all appear to be somehow related to
> > > the Windows NFSv4.1 client and not people who
> > > have been using/contributing to FreeBSD for years.
> > > This doesn't meant your opinions are not valid, but
> > > I'd like to see support for this name change include
> > > others outside your group, since I think it is basically
> > > silly. If someone wants to use this mechanism they
> > > will use it, whatever it happens to be called.)
> >
> > I think we are discussing adding an additional name as alias, not a true
> > renaming?
>
> I just had a bloody run-in with the Linux camp, who still AGGRESSIVELY
> tries to sell Linux XATTR as a solution for everything, and - of
> course - supersedes O_NAMEDATTR!! Which left me with a particularly
> bitter taste about that religious "all hail Linux XATTR" zealotism.
> Zero knowledge about ADS/O_XATTR on their side, zero logical
> arguments, but as soon as something has the letters 'ATTR' inside it
> MUST BE superseded, killed and buried by Linux XATTR!! All who oppose
> this must be slaughtered!!!!
Just to be clear, there are technical advantages to the Linux style
extended attributes (I think it was SGI Irix that first shipped a system
supporting this mechanism, but I could be incorrect w.r.t. this).
"man extattr" on FreeBSD

On FreeBSD, the zfs property xattr is set to "sa" by default.
This setting means that extended attributes are stored in a block
which handled ACLs and small extended attributes.
This is more efficient, but does not allow what you call
"named stream" to work.
--> So, if a site only wants to support ACLs and small extended
     attributes that are set atomically to a value, this is works well.

If you do want what you call "named stream", then you need to
set "xattr=dir". This makes all extended attributes, including ACLs
and small ones be stored as files in a directory. Less efficient,
but more general, in that large values (as in number of bytes) can
be stored and manipulated like any file.

It's a technical tradeoff which happens to be easy to do for
OpenZFS because it has the Solaris underpinnings in it
for "xattr=dir".

In summary, FreeBSD is a small niche market and use of
"named stream" on FreeBSD will be a small niche of that
small niche.

The simple fact that Linux has chosen to not support what
you call "named stream" guarantees that most open source
software will not use it.

>
> *Sight*
>
> I now can understand the desire to draw a clear line, and call
> O_NAMEDATTR O_NAMEDSTREAM instead.
>
> IMO an alias O_NAMEDSTREAM, _PC_NAMEDSTREAM_ENABLED,
> _PC_NAMEDSTREAM_EXISTS would be fine for me, and then let the market
> decide which to use. Please.
I will put this up on reviews.freebsd.org and see what the reviewers
think. (Personally, I still don't understand why this will matter, since no
software uses O_NAMEDSTREAM, but I do not see it as harmful, either.)
It is too late for FreeBSD 15.0, since the release cycle is in bugfix only
mode now.

rick

>
> Patch:
> --x---x---x---x--
> #define O_NAMEDSTREAM O_NAMEDATTR
> #define _PC_NAMEDSTREAM_ENABLED _PC_NAMEDATTR_ENABLED
> #define _PC_NAMEDSTREAM_EXISTS _PC_NAMEDATTR_EXISTS
> --x---x---x---x--
>
> Aurélien
> --
> Aurélien Couderc <aurelien.couderc2002@gmail.com>
> Big Data/Data mining expert, chess enthusiast
>