Re: Filesystem extended attributes and Capsicum

From: alan somers <asomers_at_gmail.com>
Date: Sat, 23 Mar 2024 13:58:37 UTC
On Fri, Mar 22, 2024 at 9:52 PM Shawn Webb <shawn.webb@hardenedbsd.org> wrote:
>
> On Fri, Mar 22, 2024 at 08:07:17PM -0600, Alan Somers wrote:
> > On Fri, Mar 22, 2024 at 6:56 PM Shawn Webb <shawn.webb@hardenedbsd.org> wrote:
> > >
> > > On Fri, Mar 22, 2024 at 06:20:48PM -0600, Alan Somers wrote:
> > > > On Fri, Mar 22, 2024 at 5:38 PM Shawn Webb <shawn.webb@hardenedbsd.org> wrote:
> > > > >
> > > > > Hey all,
> > > > >
> > > > > I'm writing an application in which I hope to enable Capsicum. I'm
> > > > > experiencing an issue whereby extattr_get_fd fails with a file
> > > > > descriptor that has all the extended attribute capabilities enabled
> > > > > (CAP_EXTATTR_DELETE, CAP_EXTATTR_GET, CAP_EXTATTR_LIST, and
> > > > > CAP_EXTATTR_SET).
> > > > >
> > > > > Looking at the kernel source (sys/kern/vfs_extattr.c) tells me that
> > > > > kern_extattr_get_fd only requires CAP_EXTATTR_GET.
> > > > >
> > > > > So I'm a bit puzzled as to why my call to extattr_get_fd(2) is
> > > > > failing. Am I doing something wrong or are filesystem extended
> > > > > attributes not supported in a Capabilities-enabled process?
> > > > >
> > > > > Here's how I'm creating the file descriptor (before calling
> > > > > cap_enter(2)):
> > > > >
> > > > > ==== BEGIN CODE ====
> > > > > static int
> > > > > open_file(const char *path)
> > > > > {
> > > > >         cap_rights_t rights;
> > > > >         int fd;
> > > > >
> > > > >         fd = open(path, O_PATH | O_CLOEXEC);
> > > > >         if (fd == -1) {
> > > > >                 return (-1);
> > > > >         }
> > > > >
> > > > >         memset(&rights, 0, sizeof(rights));
> > > > >         cap_rights_init(&rights, CAP_EXTATTR_DELETE, CAP_EXTATTR_GET,
> > > > >             CAP_EXTATTR_LIST, CAP_EXTATTR_SET);
> > > > >         cap_rights_limit(fd, &rights);
> > > > >
> > > > >         return (fd);
> > > > > }
> > > > > ==== END CODE ====
> > > > >
> > > > > Eventually, after calling cap_enter(2), the following code is called:
> > > > >
> > > > > ==== BEGIN CODE ====
> > > > > #define ATTRNAME_ENABLED "hbsd.pax.aslr"
> > > > >         sz = extattr_get_fd(fd, ctx->hc_namespace, ATTRNAME_ENABLED, NULL, 0);
> > > > >         if (sz <= 0) {
> > > > >                 if (errno == ENOATTR) {
> > > > >                         /*
> > > > >                         * This is okay, it just means that nothing has been set.
> > > > >                         * No error condition here.
> > > > >                         */
> > > > >                         return (RES_SUCCESS);
> > > > >                 }
> > > > >                 return (RES_FAIL);
> > > > >         }
> > > > > ==== END CODE ====
> > > > >
> > > > > For reference, the program's code is here:
> > > > > https://git.hardenedbsd.org/shawn.webb/hbsdctrl/-/tree/main?ref_type=heads
> > > > >
> > > > > The library code, which is what's responsible for calling the
> > > > > filesystem extended attribute related syscalls is here:
> > > > >
> > > > > https://git.hardenedbsd.org/hardenedbsd/HardenedBSD/-/tree/hardened/current/hbsdcontrol-v2/lib/libhbsdcontrol?ref_type=heads
> > > > >
> > > > > From the rights(4) manual page, I'm instructed all I need are to apply
> > > > > those capabilities to that file descriptor:
> > > > >
> > > > > ==== BEGIN PASTE ====
> > > > >      CAP_EXTATTR_DELETE   Permit extattr_delete_fd(2).
> > > > >
> > > > >      CAP_EXTATTR_GET      Permit extattr_get_fd(2).
> > > > >
> > > > >      CAP_EXTATTR_LIST     Permit extattr_list_fd(2).
> > > > >
> > > > >      CAP_EXTATTR_SET      Permit extattr_set_fd(2).
> > > > > ==== END PASTE ====
> > > > >
> > > > > So I'm a bit unsure if I'm doing something wrong.
> > > > >
> > > > > Thanks,
> > > > >
> > > > > --
> > > > > Shawn Webb
> > > > > Cofounder / Security Engineer
> > > > > HardenedBSD
> > > > >
> > > > > Tor-ified Signal: +1 303-901-1600 / shawn_webb_opsec.50
> > > > > https://git.hardenedbsd.org/hardenedbsd/pubkeys/-/raw/master/Shawn_Webb/03A4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc
> > > >
> > > > What error code does it fail with?  If it's ENOTCAPABLE, then I
> > > > suggest using dtrace to find the reason why it fails.  Do something
> > > > like this:
> > > >
> > > > dtrace -i 'fbt:kernel::return /arg1 == 93 && pid == $target/
> > > > {trace(".");}' -c ./my_application
> > > >
> > > > That will print the name of every non-inlined kernel function that
> > > > returns ENOTCAPABLE during your process.  But it will also print the
> > > > names of any other kernel functions that return an integer value of
> > > > 93.  From there, guess which function is the real source of the error.
> > > > Then you can do
> > >
> > > DTrace is unavailable on this particular system.
> > >
> > > It does indeed fail with ENOTCAPABLE. I have the kern.trap_enotcap sysctl
> > > set to 1 so that I can know at exactly what point we're failing, and
> > > it's indeed at extattr_get_fd.
> > >
> > > Thanks,
> > >
> > > --
> > > Shawn Webb
> > > Cofounder / Security Engineer
> > > HardenedBSD
> > >
> > > Tor-ified Signal: +1 303-901-1600 / shawn_webb_opsec.50
> > > https://git.hardenedbsd.org/hardenedbsd/pubkeys/-/raw/master/Shawn_Webb/03A4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc
> >
> > Without dtrace, you've got your work cut out for you.  I suggest
> > simply adding all capabilities, verifying that extattr_get_fd works,
> > and then removing capabilities until it fails.  Or, run your program
> > on vanilla FreeBSD with dtrace.
>
> HardenedBSD doesn't have any modifications that would affect Capsicum
> in this manner. Regardless, I reproduced the problem successfully on
> FreeBSD 14.0-RELEASE without any code changes. I tried running your
> DTrace script on FreeBSD 14.0-RELEASE and got this output:
>
> ==== BEGIN OUTPUT ====
> $ sudo dtrace -i 'fbt:kernel::return /arg1 == 93 && pid == $target/ {trace(".");}' -c "obj/hbsdctrl pax list /bin/ls"
> dtrace: description 'fbt:kernel::return ' matched 31396 probes
> aslr:   sysdef
> mprotect:       sysdef
> pageexec:       sysdef
> segvguard:      sysdef
> dtrace: pid 29270 has exited
> CPU     ID                    FUNCTION:NAME
>   3  47780        foffset_unlock_uio:return   .
>   3  50605              foffset_lock:return   .
>   3  47778          foffset_lock_uio:return   .
> ==== END OUTPUT ====
>
> But I'm still unsure what I'm missing, if anything.

That's red herring.  Those functions return void, but dtrace doesn't
know it.  So the "93" is just register garbage.  I also notice that
kern_extattr_get_fd isn't listed.  Are you sure that your program is
really failing with ENOTCAPABLE?  You can also try running it with
ktrace.  kdump will show you exactly what capabilities you limited the
file descriptor to.  That can help you verify if you applied the
limits correctly.
-Alan