From nobody Wed Jul 12 14:59:47 2023 X-Original-To: dev-commits-doc-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4R1LW80wfvz4mmfn for ; Wed, 12 Jul 2023 14:59:48 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4R1LW80Vljz3nCN; Wed, 12 Jul 2023 14:59:48 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1689173988; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=6Vjb127xpJBpc5ILFiAaM5xyswM1c2qWyLIfxkoVSe4=; b=YE+5OkogPL4EW5DBXqt3XJx+11qhQyieRRL2ASQ6TtLzdtkhm1EPHgNpVMhAgWXPonOckS HZqOS6CdHoX0majdJF/Ne+4uOKtII3e4MarcNcAqiGCak3X+NFQ8gkYMaZy84jGX03tTjE FKBxWmAnr5qwK0PvL106uSV4ND3tCHhCDn2+zwJ2rhsj/nBYq4StHMaRYxshF6tEn3xhle rcHRSxBokOCEenx67YTSfnMwRpKJecUtxs4XSlpuM8IAkIr91H6DNNpRZ+t4NYMVc6GjPE ZadB7HCkh/tGIUh/i878dr2ew9u9jCHvfUB6EkZ1BowXzhtcq3Ds3niUNQl+PA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1689173988; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=6Vjb127xpJBpc5ILFiAaM5xyswM1c2qWyLIfxkoVSe4=; b=I/R2gkeB7KuViJ1VjmEXosXOolUx6UF9O9cc/3x5YvU0CtlzfotDseVmDoXH/JaNoF3xHE amAeYBjwpFFhWxsz127PcF+Hcypq1hetjCaO1E7AObsycf/PZBQ4Q81mmOIOkjsUJDRocS jpv0kYYD46L9C8MB+el7otVdMivRM/eENAqa69oXQMIP2J/MD8/6hyR+7AfkbcGDO1HXcK G07moHSIyr2PH2QXJBaMQ2+JNSMxA2xMX3FJmUEZx8tiO8vCYznPhnNSIDlyMlI9gwqA0a 82vd0QmbI/SzBWRQRWm0NyZOei6ld5w0EhsGCuC/SJY8MF3Gh7gKp2SlblY/OQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1689173988; a=rsa-sha256; cv=none; b=KIYLlLKGjiPNSmooiNMowmUTcSbHC4jPHEwqpqaa6h1agBl/YII1eVdROy9rYNttFxH90A Blex8kELXnCjftrU4G/WcPY2BaS46dXX81cigHaoGo0hjFwI2d1ByA5PfAsH5BNYmNd7Ko BFUcGQRLNOdtAOE1aOL++OQVrK1ZC/EwAGLw5gwrK0kts8OLWqF7ax7D0mFNJ66r9Bd13v snyl6H8GQ+qbkjRZqAaxDZBkMDGbN2S4Ri4k5Z7YcImeFd9A/MMwxGN/KAn5Vh6c+Y3/zY g2Ymk4GSzMVBxyaC9D9s9ZNdgu/B/fNmlJyz2Ixh/T50wedlnxgOJmyJtwglYQ== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4R1LW76Yflz1Csg; Wed, 12 Jul 2023 14:59:47 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.17.1/8.17.1) with ESMTP id 36CExlIn076538; Wed, 12 Jul 2023 14:59:47 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.17.1/8.17.1/Submit) id 36CExl75076537; Wed, 12 Jul 2023 14:59:47 GMT (envelope-from git) Date: Wed, 12 Jul 2023 14:59:47 GMT Message-Id: <202307121459.36CExl75076537@gitrepo.freebsd.org> To: doc-committers@FreeBSD.org, dev-commits-doc-all@FreeBSD.org From: Lorenzo Salvadore Subject: git: 325278834f - main - Satus/2023Q2/capsicum-ktracing.adoc: Add report List-Id: Commit messages for all branches of the doc repository List-Archive: https://lists.freebsd.org/archives/dev-commits-doc-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-doc-all@freebsd.org X-BeenThere: dev-commits-doc-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: salvadore X-Git-Repository: doc X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 325278834f31e1ab3cdc017d8996d80212f15a67 Auto-Submitted: auto-generated X-ThisMailContainsUnwantedMimeParts: N The branch main has been updated by salvadore: URL: https://cgit.FreeBSD.org/doc/commit/?id=325278834f31e1ab3cdc017d8996d80212f15a67 commit 325278834f31e1ab3cdc017d8996d80212f15a67 Author: Jake Freeland AuthorDate: 2023-07-12 14:54:23 +0000 Commit: Lorenzo Salvadore CommitDate: 2023-07-12 14:58:59 +0000 Satus/2023Q2/capsicum-ktracing.adoc: Add report Reviewed by: status (Pau Amma ) Approved by: dbaio (mentor, implicit) Pull Request: https://github.com/freebsd/freebsd-doc/pull/196 --- .../report-2023-04-2023-06/capsicum-ktracing.adoc | 230 +++++++++++++++++++++ 1 file changed, 230 insertions(+) diff --git a/website/content/en/status/report-2023-04-2023-06/capsicum-ktracing.adoc b/website/content/en/status/report-2023-04-2023-06/capsicum-ktracing.adoc new file mode 100644 index 0000000000..463852075d --- /dev/null +++ b/website/content/en/status/report-2023-04-2023-06/capsicum-ktracing.adoc @@ -0,0 +1,230 @@ +=== Security Sandboxing Using man:ktrace[1] + +Links: + +link:https://github.com/jakesfreeland/freebsd-src/tree/ff/ktrace[ktrace branch] URL: link:https://github.com/jakesfreeland/freebsd-src/tree/ff/ktrace[] + + +Contact: Jake Freeland + +==== Capsicumization With man:ktrace[1] + +This report introduces an extension to man:ktrace[1] that logs capability violations for programs that have not been Capsicumized. + +The first logical step in Capsicumization is determining where your program is raising capability violations. +You could approach this issue by looking through the source and removing Capsicum-incompatible code, but this can be tedious and requires the developer to be familiar with everything that is not allowed in capability mode. + +An alternative to finding violations manually is to use man:ktrace[1]. +The man:ktrace[1] utility logs kernel activity for a specified process. +Capsicum violations occur inside of the kernel, so man:ktrace[1] can record and return extra information about your program's violations with the `-t p` option. + +Programs traditionally need to be put into capability mode before they will report violations. +When a restricted system call is entered, it will fail and return with `ECAPMODE: Not permitted in capability mode`. +If the developer is doing error checking, then it is likely that their program will terminate with that error. +This behavior made violation tracing inconvenient because man:ktrace[1] would only report the first capability violation, and then the program would terminate. + +Luckily, a new extension to man:ktrace[1] can record violations when a program is **NOT** in capability mode. +This means that any developer can run capability violation tracing on their program with no modification to see where it is raising violations. +Since the program is never actually put into capability mode, it will still acquire resources and execute normally. + +==== Violation Tracing Examples + +The `cap_violate` program, shown below, attempts to raise every type of violation that man:ktrace[1] can capture: + +[source, shell] +---- +# ktrace -t p ./cap_violate +# kdump +1603 ktrace CAP system call not allowed: execve +1603 foo CAP system call not allowed: open +1603 foo CAP system call not allowed: open +1603 foo CAP system call not allowed: open +1603 foo CAP system call not allowed: open +1603 foo CAP system call not allowed: readlink +1603 foo CAP system call not allowed: open +1603 foo CAP cpuset_setaffinity: restricted cpuset operation +1603 foo CAP openat: restricted VFS lookup: AT_FDCWD +1603 foo CAP openat: restricted VFS lookup: / +1603 foo CAP system call not allowed: bind +1603 foo CAP sendto: restricted address lookup: struct sockaddr { AF_INET, 0.0.0.0:5000 } +1603 foo CAP socket: protocol not allowed: IPPROTO_ICMP +1603 foo CAP kill: signal delivery not allowed: SIGCONT +1603 foo CAP system call not allowed: chdir +1603 foo CAP system call not allowed: fcntl, cmd: F_KINFO +1603 foo CAP operation requires CAP_WRITE, descriptor holds CAP_READ +1603 foo CAP attempt to increase capabilities from CAP_READ to CAP_READ,CAP_WRITE +---- + +The first 7 `system call not allowed` entries did not explicitly originate from the `cap_violate` program code. +Instead, they were raised by FreeBSD's C runtime libraries. +This becomes apparent when you trace namei translations alongside capability violations using the `-t np` option: + +[source, shell] +---- +# ktrace -t np ./cap_violate +# kdump +1632 ktrace CAP system call not allowed: execve +1632 ktrace NAMI "./cap_violate" +1632 ktrace NAMI "/libexec/ld-elf.so.1" +1632 foo CAP system call not allowed: open +1632 foo NAMI "/etc/libmap.conf" +1632 foo CAP system call not allowed: open +1632 foo NAMI "/usr/local/etc/libmap.d" +1632 foo CAP system call not allowed: open +1632 foo NAMI "/var/run/ld-elf.so.hints" +1632 foo CAP system call not allowed: open +1632 foo NAMI "/lib/libc.so.7" +1632 foo CAP system call not allowed: readlink +1632 foo NAMI "/etc/malloc.conf" +1632 foo CAP system call not allowed: open +1632 foo NAMI "/dev/pvclock" +1632 foo CAP cpuset_setaffinity: restricted cpuset operation +1632 foo NAMI "ktrace.out" +1632 foo CAP openat: restricted VFS lookup: AT_FDCWD +1632 foo NAMI "/" +1632 foo CAP openat: restricted VFS lookup: / +1632 foo CAP system call not allowed: bind +1632 foo CAP sendto: restricted address lookup: struct sockaddr { AF_INET, 0.0.0.0:5000 } +1632 foo CAP socket: protocol not allowed: IPPROTO_ICMP +1632 foo CAP kill: signal delivery not allowed: SIGCONT +1632 foo CAP system call not allowed: chdir +1632 foo NAMI "." +1632 foo CAP system call not allowed: fcntl, cmd: F_KINFO +1632 foo CAP operation requires CAP_WRITE, descriptor holds CAP_READ +1632 foo CAP attempt to increase capabilities from CAP_READ to CAP_READ,CAP_WRITE +---- + +In practice, capability mode is always entered following the initialization of the C runtime libraries, so a program would never trigger those those first 7 violations. +We are only seeing them because man:ktrace[1] starts recording violations before the program starts. + +This demonstration makes it clear that violation tracing is not always perfect. +It is a helpful guide for detecting restricted system calls, but may not always parody your program's actual behavior in capability mode. +In capability mode, violations are equivalent to errors; they are an indication to stop execution. +Violation tracing is ignoring this suggestion and continuing execution anyway, so invalid violations may be reported. + +The next example traces violations from the man:unzip[1] utility (pre-Capsicumization): + +[source, shell] +---- +# ktrace -t np unzip foo.zip +Archive: foo.zip +creating: bar/ +extracting: bar/bar.txt +creating: baz/ +extracting: baz/baz.txt +# kdump +1926 ktrace CAP system call not allowed: execve +1926 ktrace NAMI "/usr/bin/unzip" +1926 ktrace NAMI "/libexec/ld-elf.so.1" +1926 unzip CAP system call not allowed: open +1926 unzip NAMI "/etc/libmap.conf" +1926 unzip CAP system call not allowed: open +1926 unzip NAMI "/usr/local/etc/libmap.d" +1926 unzip CAP system call not allowed: open +1926 unzip NAMI "/var/run/ld-elf.so.hints" +1926 unzip CAP system call not allowed: open +1926 unzip NAMI "/lib/libarchive.so.7" +1926 unzip CAP system call not allowed: open +1926 unzip NAMI "/usr/lib/libarchive.so.7" +1926 unzip CAP system call not allowed: open +1926 unzip NAMI "/lib/libc.so.7" +1926 unzip CAP system call not allowed: open +1926 unzip NAMI "/lib/libz.so.6" +1926 unzip CAP system call not allowed: open +1926 unzip NAMI "/lib/libbz2.so.4" +1926 unzip CAP system call not allowed: open +1926 unzip NAMI "/usr/lib/libbz2.so.4" +1926 unzip CAP system call not allowed: open +1926 unzip NAMI "/lib/liblzma.so.5" +1926 unzip CAP system call not allowed: open +1926 unzip NAMI "/usr/lib/liblzma.so.5" +1926 unzip CAP system call not allowed: open +1926 unzip NAMI "/lib/libbsdxml.so.4" +1926 unzip CAP system call not allowed: open +1926 unzip NAMI "/lib/libprivatezstd.so.5" +1926 unzip CAP system call not allowed: open +1926 unzip NAMI "/usr/lib/libprivatezstd.so.5" +1926 unzip CAP system call not allowed: open +1926 unzip NAMI "/lib/libcrypto.so.111" +1926 unzip CAP system call not allowed: open +1926 unzip NAMI "/lib/libmd.so.6" +1926 unzip CAP system call not allowed: open +1926 unzip NAMI "/lib/libthr.so.3" +1926 unzip CAP system call not allowed: readlink +1926 unzip NAMI "/etc/malloc.conf" +1926 unzip CAP system call not allowed: open +1926 unzip NAMI "/dev/pvclock" +1926 unzip NAMI "foo.zip" +1926 unzip CAP openat: restricted VFS lookup: AT_FDCWD +1926 unzip CAP system call not allowed: open +1926 unzip NAMI "/etc/localtime" +1926 unzip NAMI "bar" +1926 unzip CAP fstatat: restricted VFS lookup: AT_FDCWD +1926 unzip CAP system call not allowed: mkdir +1926 unzip NAMI "bar" +1926 unzip NAMI "bar" +1926 unzip CAP fstatat: restricted VFS lookup: AT_FDCWD +1926 unzip NAMI "bar/bar.txt" +1926 unzip CAP fstatat: restricted VFS lookup: AT_FDCWD +1926 unzip NAMI "bar/bar.txt" +1926 unzip CAP openat: restricted VFS lookup: AT_FDCWD +1926 unzip NAMI "baz" +1926 unzip CAP fstatat: restricted VFS lookup: AT_FDCWD +1926 unzip CAP system call not allowed: mkdir +1926 unzip NAMI "baz" +1926 unzip NAMI "baz" +1926 unzip CAP fstatat: restricted VFS lookup: AT_FDCWD +1926 unzip NAMI "baz/baz.txt" +1926 unzip CAP fstatat: restricted VFS lookup: AT_FDCWD +1926 unzip NAMI "baz/baz.txt" +1926 unzip CAP openat: restricted VFS lookup: AT_FDCWD +---- + +The violation tracing output for man:unzip[1] is more akin to what a developer would see when tracing their own program for the first time. +Most programs link against libraries. +In this case, man:unzip[1] is linking against man:libarchive[3], which is reflected here: + +[source, shell] +---- +1926 unzip CAP system call not allowed: open +1926 unzip NAMI "/lib/libarchive.so.7" +1926 unzip CAP system call not allowed: open +1926 unzip NAMI "/usr/lib/libarchive.so.7" +---- + +The violations for man:unzip[1] can be found below the C runtime violations: + +[source, shell] +---- +1926 unzip NAMI "foo.zip" +1926 unzip CAP openat: restricted VFS lookup: AT_FDCWD +1926 unzip CAP system call not allowed: open +1926 unzip NAMI "/etc/localtime" +1926 unzip NAMI "bar" +1926 unzip CAP fstatat: restricted VFS lookup: AT_FDCWD +1926 unzip CAP system call not allowed: mkdir +1926 unzip NAMI "bar" +1926 unzip NAMI "bar" +1926 unzip CAP fstatat: restricted VFS lookup: AT_FDCWD +1926 unzip NAMI "bar/bar.txt" +1926 unzip CAP fstatat: restricted VFS lookup: AT_FDCWD +1926 unzip NAMI "bar/bar.txt" +1926 unzip CAP openat: restricted VFS lookup: AT_FDCWD +1926 unzip NAMI "baz" +1926 unzip CAP fstatat: restricted VFS lookup: AT_FDCWD +1926 unzip CAP system call not allowed: mkdir +1926 unzip NAMI "baz" +1926 unzip NAMI "baz" +1926 unzip CAP fstatat: restricted VFS lookup: AT_FDCWD +1926 unzip NAMI "baz/baz.txt" +1926 unzip CAP fstatat: restricted VFS lookup: AT_FDCWD +1926 unzip NAMI "baz/baz.txt" +1926 unzip CAP openat: restricted VFS lookup: AT_FDCWD +---- + +In this instance, man:unzip[1] is recreating the file structure contained in the zip archive. +Violations are being raised because the `AT_FDCWD` value cannot be used in capability mode. +The bulk of these violations can be fixed by opening `AT_FDCWD` (the current directory) before entering capability mode and passing that descriptor into man:openat[2], man:fstatat[2], and man:mkdirat[2] as a relative reference. + +Violation tracing may not automatically Capsicumize programs, but it is another tool in the developer's toolbox. +It only takes a few seconds to run a program under man:ktrace[1] and the result is almost always a decent starting point for sandboxing your program using Capsicum. + +Sponsor: FreeBSD Foundation