From nobody Wed Aug 13 22:53:10 2025 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4c2P4M0R58z64dyF for ; Wed, 13 Aug 2025 23:01:07 +0000 (UTC) (envelope-from jrtc27@jrtc27.com) Received: from mail-lf1-f47.google.com (mail-lf1-f47.google.com [209.85.167.47]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4c2P4L4wrkz3h8D for ; Wed, 13 Aug 2025 23:01:06 +0000 (UTC) (envelope-from jrtc27@jrtc27.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-lf1-f47.google.com with SMTP id 2adb3069b0e04-55ce521f77bso308081e87.1 for ; Wed, 13 Aug 2025 16:01:06 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1755126065; x=1755730865; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=xNYTLIL06ify0ucbOLLuZeBFAlehj1IWwIB2D4OsuZc=; b=ssc64xVMvh7BazhRVGf7UZNqteyzghVMh6qEtPANN9gULsAcy0TDMUJ1TcI2Rdnpyo d+DNHt3UpQc2FOOZuIvIrdOk7P59nnyhMh9iNj6LJBXYK9+JQyMjaAzgErCEmfb6AXAj OC5fDwN+mh0O8Yt2IbqVq3r9k6bR/XxV1rNkM0YoUlQ7H8yLkbMGSsRH7ohmDHtsn0+U i5OuFtQMkS6e/S1Y/JbgogR4Wjih78yBZ2WAevbin2dq7gRYjK0tNiD6mZXMayLUNe2s MCKG1epYLZg5Coztd3n66F6MD4PU9liCtS1k97ff07GhG7XfiAZ1jxQLZWNyqlxXNHmK oovw== X-Forwarded-Encrypted: i=1; AJvYcCXsXf0esKV+AXAX6arhPmm0olEZ7YHXyefhjaCaqqjYkJ5cMtcK9KurPgUhA/R6/DLw0RVFBZxhpQ/eirQFyy4L5dTT8g==@freebsd.org X-Gm-Message-State: AOJu0YzMdykHh9n4k7uWyiLiPSiu3Fzj8cOWsQPnH4u5/jQ9YakpjWiM 4Qby0vblDTaXJBLEtugvX9lPDDiJkc7qL8hurXd0FPT7bbELGdbT1oqKk4Hh7Q0NdTuFgV0RdwC VA4nZ X-Gm-Gg: ASbGnct6ItaU+YcQLOe7QyaY2dTd26CQWKdZ039A+zyWD/k8+ezCS5YBaLqfMmdgOd0 axFnN6187zHDJovkEHZ2W5k6pvQ2hkH172WjpxqvbY2AvsUMCrwdvYxqmedxv1VRpUbnco58eYT 9DnPYWh5h0QMfuh3PMPWSVsDtq0wRn+J3GS5yIpfMiPz/yfWV48SpWOjAJqSu6vMEBIpm7lneIx 4VlFXEpslusnlr8fRbhZ9yIIC9Q/aDF7Um/krAkQAGPp0djOyW6TupFNPJgZeM5zGwYhXXIy48H YUhhTM5RS640mbEDGQRRgMS6YuvAhoq4hXXf1p1n8FKBMrFoes3Y1g+t233K95ZJ92bT7yIxNl2 iHcBHIjaXvFrwqIA2+VqxDGo7Q/FVkfNm/ldToEJFQRrVYfdO+T6hapVYHOCwT1FKh4k7 X-Google-Smtp-Source: AGHT+IHuyl7lNUkrxvjpzBB+G3wn7d7nGlvlyncEs4qVQ6Z0o4v3NtpxDG0OjGcpE+hpEQB3v69Kng== X-Received: by 2002:a05:600c:a06:b0:456:1d61:b0f2 with SMTP id 5b1f17b1804b1-45a1b687253mr4062295e9.30.1755125603591; Wed, 13 Aug 2025 15:53:23 -0700 (PDT) Received: from smtpclient.apple (global-184-7.n-1.net.cam.ac.uk. [131.111.184.7]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-45a1a50a42asm16908215e9.1.2025.08.13.15.53.22 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 13 Aug 2025 15:53:22 -0700 (PDT) Content-Type: text/plain; charset=utf-8 List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-main@freebsd.org Sender: owner-dev-commits-src-main@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3826.600.51.1.1\)) Subject: Re: git: 81d8827ad875 - main - certctl: Reimplement in C From: Jessica Clarke In-Reply-To: <202508132225.57DMPlLZ083341@gitrepo.freebsd.org> Date: Wed, 13 Aug 2025 23:53:10 +0100 Cc: "src-committers@freebsd.org" , "dev-commits-src-all@freebsd.org" , "dev-commits-src-main@freebsd.org" Content-Transfer-Encoding: quoted-printable Message-Id: References: <202508132225.57DMPlLZ083341@gitrepo.freebsd.org> To: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= X-Mailer: Apple Mail (2.3826.600.51.1.1) X-Rspamd-Queue-Id: 4c2P4L4wrkz3h8D X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US] On 13 Aug 2025, at 23:25, Dag-Erling Sm=C3=B8rgrav = wrote: >=20 > The branch main has been updated by des: >=20 > URL: = https://cgit.FreeBSD.org/src/commit/?id=3D81d8827ad8752e35411204541f1f09df= 1481e417 >=20 > commit 81d8827ad8752e35411204541f1f09df1481e417 > Author: Dag-Erling Sm=C3=B8rgrav > AuthorDate: 2025-08-13 22:25:27 +0000 > Commit: Dag-Erling Sm=C3=B8rgrav > CommitDate: 2025-08-13 22:25:27 +0000 >=20 > certctl: Reimplement in C >=20 > Notable changes include: >=20 > * We no longer forget manually untrusted certificates when = rehashing. >=20 > * Rehash will now scan the existing directory and progressively = replace > its contents with those of the new trust store. The trust store = as a > whole is not replaced atomically, but each file within it is. >=20 > * We no longer attempt to link to the original files, but we don't = copy > them either. Instead, we write each certificate out in its = minimal > form. >=20 > * We now generate a trust bundle in addition to the hashed = diretory. > This also contains only the minimal DER form of each certificate. >=20 > * The C version is approximately two orders of magnitude faster = than the > sh version, with rehash taking ~100 ms vs ~5-25 s depending on = whether > ca_root_nss is installed. >=20 > * The DISTBASE concept has been dropped; the same effect can be = achieved > by adjusting DESTDIR. That=E2=80=99s not quite true. DISTBASE was separate from DESTDIR = because the expectation of distributeworld is that there is a single METALOG file for all of the distribution sets combined, where each line in the METALOG includes the distribution set=E2=80=99s directory name. Have you verified that distributeworld -DNO_ROOT (as is now the only supported option for release builds) works correctly and includes all the hashed certs? See 232cf6be4bc4 ("certctl: Introduce a new -d option=E2=80=9D)= for the rationale behind why I introduced it as a separate option in the first place; prior to that there was just DESTDIR that pointed at the distribution=E2=80=99s subdirectory. As mentioned on IRC this also breaks the macOS cross-build due to not being able to find OpenSSL headers. Jessica > * We now also have rudimentary tests. >=20 > Reviewed by: kevans > Differential Revision: https://reviews.freebsd.org/D42320 > --- > Makefile.inc1 | 21 +- > usr.sbin/certctl/Makefile | 7 +- > usr.sbin/certctl/certctl.8 | 94 +-- > usr.sbin/certctl/certctl.c | 1060 = ++++++++++++++++++++++++++++++++ > usr.sbin/certctl/certctl.sh | 366 ----------- > usr.sbin/certctl/tests/Makefile | 5 + > usr.sbin/certctl/tests/certctl.subr | 44 ++ > usr.sbin/certctl/tests/certctl_test.sh | 221 +++++++ > 8 files changed, 1404 insertions(+), 414 deletions(-) >=20 > diff --git a/Makefile.inc1 b/Makefile.inc1 > index 9128d1d8ee77..e67bc7f5d1b1 100644 > --- a/Makefile.inc1 > +++ b/Makefile.inc1 > @@ -1021,8 +1021,7 @@ IMAKE_MTREE=3D MTREE_CMD=3D"${MTREE_CMD} = ${MTREEFLAGS}" > .endif >=20 > .if make(distributeworld) > -CERTCTLDESTDIR=3D ${DESTDIR}/${DISTDIR} > -CERTCTLFLAGS+=3D -d /base > +CERTCTLDESTDIR=3D ${DESTDIR}/${DISTDIR}/base > .else > CERTCTLDESTDIR=3D ${DESTDIR} > .endif > @@ -1541,14 +1540,10 @@ distributeworld installworld stageworld: = _installcheck_world .PHONY > .endif # make(distributeworld) > ${_+_}cd ${.CURDIR}; ${IMAKE} re${.TARGET:S/world$//}; \ > ${IMAKEENV} rm -rf ${INSTALLTMP} > -.if !make(packageworld) && ${MK_CAROOT} !=3D "no" > - @if which openssl>/dev/null; then \ > - PATH=3D${TMPPATH:Q}:${PATH:Q} \ > - LOCALBASE=3D${LOCALBASE:Q} \ > - sh ${SRCTOP}/usr.sbin/certctl/certctl.sh ${CERTCTLFLAGS} rehash; = \ > - else \ > - echo "No openssl on the host, not rehashing certificates target -- = /etc/ssl may not be populated."; \ > - fi > +.if !make(packageworld) && ${MK_CAROOT} !=3D "no" && ${MK_OPENSSL} !=3D= "no" > + PATH=3D${TMPPATH:Q}:${PATH:Q} \ > + LOCALBASE=3D${LOCALBASE:Q} \ > + certctl ${CERTCTLFLAGS} rehash > .endif > .if make(distributeworld) > .for dist in ${EXTRA_DISTRIBUTIONS} > @@ -2712,6 +2707,11 @@ _basic_bootstrap_tools+=3Dsbin/md5 > _basic_bootstrap_tools+=3Dusr.sbin/tzsetup > .endif >=20 > +# certctl is needed as an install tool > +.if ${MK_CAROOT} !=3D "no" && ${MK_OPENSSL} !=3D "no" > +_certctl=3Dusr.sbin/certctl > +.endif > + > .if defined(BOOTSTRAP_ALL_TOOLS) > _other_bootstrap_tools+=3D${_basic_bootstrap_tools} > .for _subdir _links in ${_basic_bootstrap_tools_multilink} > @@ -2775,6 +2775,7 @@ bootstrap-tools: ${_bt}-links .PHONY > ${_strfile} \ > usr.bin/dtc \ > ${_cat} \ > + ${_certctl} \ > ${_kbdcontrol} \ > ${_elftoolchain_libs} \ > ${_libkldelf} \ > diff --git a/usr.sbin/certctl/Makefile b/usr.sbin/certctl/Makefile > index 88c024daf7e6..5430dbf24853 100644 > --- a/usr.sbin/certctl/Makefile > +++ b/usr.sbin/certctl/Makefile > @@ -1,5 +1,10 @@ > +.include > + > PACKAGE=3D certctl > -SCRIPTS=3Dcertctl.sh > +PROG=3D certctl > MAN=3D certctl.8 > +LIBADD=3D crypto > +HAS_TESTS=3D > +SUBDIR.${MK_TESTS}=3D tests >=20 > .include > diff --git a/usr.sbin/certctl/certctl.8 b/usr.sbin/certctl/certctl.8 > index 7e49bb89e2ac..97bdc840c359 100644 > --- a/usr.sbin/certctl/certctl.8 > +++ b/usr.sbin/certctl/certctl.8 > @@ -24,7 +24,7 @@ > .\" IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE > .\" POSSIBILITY OF SUCH DAMAGE. > .\" > -.Dd July 17, 2025 > +.Dd August 11, 2025 > .Dt CERTCTL 8 > .Os > .Sh NAME > @@ -32,63 +32,83 @@ > .Nd "tool for managing trusted and untrusted TLS certificates" > .Sh SYNOPSIS > .Nm > -.Op Fl v > +.Op Fl lv > .Ic list > .Nm > -.Op Fl v > +.Op Fl lv > .Ic untrusted > .Nm > -.Op Fl cnUv > +.Op Fl BnUv > .Op Fl D Ar destdir > .Op Fl M Ar metalog > .Ic rehash > .Nm > -.Op Fl cnv > -.Ic untrust Ar file > +.Op Fl nv > +.Ic untrust Ar > .Nm > -.Op Fl cnv > -.Ic trust Ar file > +.Op Fl nv > +.Ic trust Ar > .Sh DESCRIPTION > The > .Nm > utility manages the list of TLS Certificate Authorities that are = trusted by > applications that use OpenSSL. > .Pp > -Flags: > +The following options are available: > .Bl -tag -width 4n > -.It Fl c > -Copy certificates instead of linking to them. > +.It Fl B > +Do not generate a bundle. > +This option is only valid in conjunction with the > +.Ic rehash > +command. > .It Fl D Ar destdir > Specify the DESTDIR (overriding values from the environment). > -.It Fl d Ar distbase > -Specify the DISTBASE (overriding values from the environment). > +.It Fl l > +When listing installed (trusted or untrusted) certificates, show the > +full path and distinguished name for each certificate. > .It Fl M Ar metalog > -Specify the path of the METALOG file (default: $DESTDIR/METALOG). > +Specify the path of the METALOG file > +.Po > +default: > +.Pa ${DESTDIR}/METALOG > +.Pc . > +This option is only valid in conjunction with the > +.Ic rehash > +command. > .It Fl n > -No-Op mode, do not actually perform any actions. > +Dry-run mode. > +Do not actually perform any actions except write the metalog. > .It Fl v > -Be verbose, print details about actions before performing them. > +Verbose mode. > +Print detailed information about each action taken. > .It Fl U > -Unprivileged mode, do not change the ownership of created links. > -Do record the ownership in the METALOG file. > +Unprivileged mode. > +Do not attempt to set the ownership of created files. > +This option is only valid in conjunction with the > +.Fl M > +option and the > +.Ic rehash > +command. > .El > .Pp > Primary command functions: > .Bl -tag -width untrusted > .It Ic list > -List all currently trusted certificate authorities. > +List all currently trusted certificates. > .It Ic untrusted > List all currently untrusted certificates. > .It Ic rehash > -Rebuild the list of trusted certificate authorities by scanning all = directories > +Rebuild the list of trusted certificates by scanning all directories > in > .Ev TRUSTPATH > and all untrusted certificates in > .Ev UNTRUSTPATH . > -A symbolic link to each trusted certificate is placed in > +A copy of each trusted certificate is placed in > .Ev CERTDESTDIR > and each untrusted certificate in > .Ev UNTRUSTDESTDIR . > +In addition, a bundle containing the trusted certificates is placed = in > +.Ev BUNDLEFILE . > .It Ic untrust > Add the specified file to the untrusted list. > .It Ic trust > @@ -98,8 +118,6 @@ Remove the specified file from the untrusted list. > .Bl -tag -width UNTRUSTDESTDIR > .It Ev DESTDIR > Alternate destination directory to operate on. > -.It Ev DISTBASE > -Additional path component to include when operating on certificate = directories. > .It Ev LOCALBASE > Location for local programs. > Defaults to the value of the user.localbase sysctl which is usually > @@ -107,32 +125,34 @@ Defaults to the value of the user.localbase = sysctl which is usually > .It Ev TRUSTPATH > List of paths to search for trusted certificates. > Default: > -.Pa /usr/share/certs/trusted > -.Pa /usr/local/share/certs > -.Pa /etc/ssl/certs > +.Pa ${DESTDIR}/usr/share/certs/trusted > +.Pa ${DESTDIR}${LOCALBASE}/share/certs/trusted > +.Pa ${DESTDIR}${LOCALBASE}/share/certs > .It Ev UNTRUSTPATH > List of paths to search for untrusted certificates. > Default: > -.Pa /usr/share/certs/untrusted > -.Pa /etc/ssl/untrusted > -.Pa /etc/ssl/blacklisted > -.It Ev CERTDESTDIR > +.Pa ${DESTDIR}/usr/share/certs/untrusted > +.Pa ${DESTDIR}${LOCALBASE}/share/certs/untrusted > +.It Ev TRUSTDESTDIR > Destination directory for symbolic links to trusted certificates. > Default: > -.Pa /etc/ssl/certs > +.Pa ${DESTDIR}/etc/ssl/certs > .It Ev UNTRUSTDESTDIR > Destination directory for symbolic links to untrusted certificates. > Default: > -.Pa /etc/ssl/untrusted > -.It Ev EXTENSIONS > -List of file extensions to read as certificate files. > -Default: *.pem *.crt *.cer *.crl *.0 > +.Pa ${DESTDIR}/etc/ssl/untrusted > +.It Ev BUNDLE > +File name of bundle to produce. > .El > .Sh SEE ALSO > .Xr openssl 1 > .Sh HISTORY > .Nm > first appeared in > -.Fx 12.2 > +.Fx 12.2 . > .Sh AUTHORS > -.An Allan Jude Aq Mt allanjude@freebsd.org > +.An -nosplit > +The original shell implementation was written by > +.An Allan Jude Aq Mt allanjude@FreeBSD.org . > +The current C implementation was written by > +.An Dag-Erling Sm\(/orgrav Aq Mt des@FreeBSD.org . > diff --git a/usr.sbin/certctl/certctl.c b/usr.sbin/certctl/certctl.c > new file mode 100644 > index 000000000000..6687e56f23b4 > --- /dev/null > +++ b/usr.sbin/certctl/certctl.c > @@ -0,0 +1,1060 @@ > +/*- > + * Copyright (c) 2023-2025 Dag-Erling Sm=C3=B8rgrav > + * > + * SPDX-License-Identifier: BSD-2-Clause > + */ > + > +#include > +#include > +#include > + > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > + > +#include > + > +#define info(fmt, ...) \ > + do { \ > + if (verbose) \ > + fprintf(stderr, fmt "\n", ##__VA_ARGS__); \ > + } while (0) > + > +static char * > +xasprintf(const char *fmt, ...) > +{ > + va_list ap; > + char *str; > + int ret; > + > + va_start(ap, fmt); > + ret =3D vasprintf(&str, fmt, ap); > + va_end(ap); > + if (ret < 0 || str =3D=3D NULL) > + err(1, NULL); > + return (str); > +} > + > +static char * > +xstrdup(const char *str) > +{ > + char *dup; > + > + if ((dup =3D strdup(str)) =3D=3D NULL) > + err(1, NULL); > + return (dup); > +} > + > +static void usage(void); > + > +static bool dryrun; > +static bool longnames; > +static bool nobundle; > +static bool unprivileged; > +static bool verbose; > + > +static const char *localbase; > +static const char *destdir; > +static const char *metalog; > + > +static const char *uname =3D "root"; > +static const char *gname =3D "wheel"; > + > +static const char *const default_trusted_paths[] =3D { > + "/usr/share/certs/trusted", > + "%L/share/certs/trusted", > + "%L/share/certs", > + NULL > +}; > +static char **trusted_paths; > + > +static const char *const default_untrusted_paths[] =3D { > + "/usr/share/certs/untrusted", > + "%L/share/certs/untrusted", > + NULL > +}; > +static char **untrusted_paths; > + > +static char *trusted_dest; > +static char *untrusted_dest; > +static char *bundle_dest; > + > +#define SSL_PATH "/etc/ssl" > +#define TRUSTED_DIR "certs" > +#define TRUSTED_PATH SSL_PATH "/" TRUSTED_DIR > +#define UNTRUSTED_DIR "untrusted" > +#define UNTRUSTED_PATH SSL_PATH "/" UNTRUSTED_DIR > +#define LEGACY_DIR "blacklisted" > +#define LEGACY_PATH SSL_PATH "/" LEGACY_DIR > +#define BUNDLE_FILE "cert.pem" > +#define BUNDLE_PATH SSL_PATH "/" BUNDLE_FILE > + > +static FILE *mlf; > + > +/* > + * Split a colon-separated list into a NULL-terminated array. > + */ > +static char ** > +split_paths(const char *str) > +{ > + char **paths; > + const char *p, *q; > + unsigned int i, n; > + > + for (p =3D str, n =3D 1; *p; p++) { > + if (*p =3D=3D ':') > + n++; > + } > + if ((paths =3D calloc(n + 1, sizeof(*paths))) =3D=3D NULL) > + err(1, NULL); > + for (p =3D q =3D str, i =3D 0; i < n; i++, p =3D q + 1) { > + q =3D strchrnul(p, ':'); > + if ((paths[i] =3D strndup(p, q - p)) =3D=3D NULL) > + err(1, NULL); > + } > + return (paths); > +} > + > +/* > + * Expand %L into LOCALBASE and prefix DESTDIR. > + */ > +static char * > +expand_path(const char *template) > +{ > + if (template[0] =3D=3D '%' && template[1] =3D=3D 'L') > + return (xasprintf("%s%s%s", destdir, localbase, template + 2)); > + return (xasprintf("%s%s", destdir, template)); > +} > + > +/* > + * Expand an array of paths. > + */ > +static char ** > +expand_paths(const char *const *templates) > +{ > + char **paths; > + unsigned int i, n; > + > + for (n =3D 0; templates[n] !=3D NULL; n++) > + continue; > + if ((paths =3D calloc(n + 1, sizeof(*paths))) =3D=3D NULL) > + err(1, NULL); > + for (i =3D 0; i < n; i++) > + paths[i] =3D expand_path(templates[i]); > + return (paths); > +} > + > +/* > + * If destdir is a prefix of path, returns a pointer to the rest of = path, > + * otherwise returns path. > + */ > +static const char * > +unexpand_path(const char *path) > +{ > + const char *p =3D path; > + const char *q =3D destdir; > + > + while (*p && *p =3D=3D *q) { > + p++; > + q++; > + } > + return (*q =3D=3D '\0' && *p =3D=3D '/' ? p : path); > +} > + > +/* > + * X509 certificate in a rank-balanced tree. > + */ > +struct cert { > + RB_ENTRY(cert) entry; > + unsigned long hash; > + char *name; > + X509 *x509; > + char *path; > +}; > + > +static void > +free_cert(struct cert *cert) > +{ > + free(cert->name); > + X509_free(cert->x509); > + free(cert->path); > + free(cert); > +} > + > +static int > +certcmp(const struct cert *a, const struct cert *b) > +{ > + return (X509_cmp(a->x509, b->x509)); > +} > + > +RB_HEAD(cert_tree, cert); > +static struct cert_tree trusted =3D RB_INITIALIZER(&trusted); > +static struct cert_tree untrusted =3D RB_INITIALIZER(&untrusted); > +RB_GENERATE_STATIC(cert_tree, cert, entry, certcmp); > + > +static void > +free_certs(struct cert_tree *tree) > +{ > + struct cert *cert, *tmp; > + > + RB_FOREACH_SAFE(cert, cert_tree, tree, tmp) { > + RB_REMOVE(cert_tree, tree, cert); > + free_cert(cert); > + } > +} > + > +static struct cert * > +find_cert(struct cert_tree *haystack, X509 *x509) > +{ > + struct cert needle =3D { .x509 =3D x509 }; > + > + return (RB_FIND(cert_tree, haystack, &needle)); > +} > + > +/* > + * File containing a certificate in a rank-balanced tree sorted by > + * certificate hash and disambiguating counter. This is needed = because > + * the certificate hash function is prone to collisions, = necessitating a > + * counter to distinguish certificates that hash to the same value. > + */ > +struct file { > + RB_ENTRY(file) entry; > + const struct cert *cert; > + unsigned int c; > +}; > + > +static int > +filecmp(const struct file *a, const struct file *b) > +{ > + if (a->cert->hash > b->cert->hash) > + return (1); > + if (a->cert->hash < b->cert->hash) > + return (-1); > + return (a->c - b->c); > +} > + > +RB_HEAD(file_tree, file); > +RB_GENERATE_STATIC(file_tree, file, entry, filecmp); > + > +/* > + * Lexicographical sort for scandir(). > + */ > +static int > +lexisort(const struct dirent **d1, const struct dirent **d2) > +{ > + return (strcmp((*d1)->d_name, (*d2)->d_name)); > +} > + > +/* > + * Read certificate(s) from a single file and insert them into a = tree. > + * Ignore certificates that already exist in the tree. If exclude is = not > + * null, also ignore certificates that exist in exclude. > + * > + * Returns the number certificates added to the tree, or -1 on = failure. > + */ > +static int > +read_cert(const char *path, struct cert_tree *tree, struct cert_tree = *exclude) > +{ > + FILE *f; > + X509 *x509; > + X509_NAME *name; > + struct cert *cert; > + unsigned long hash; > + int ni, no; > + > + if ((f =3D fopen(path, "r")) =3D=3D NULL) { > + warn("%s", path); > + return (-1); > + } > + for (ni =3D no =3D 0; > + (x509 =3D PEM_read_X509(f, NULL, NULL, NULL)) !=3D NULL; > + ni++) { > + hash =3D X509_subject_name_hash(x509); > + if (exclude && find_cert(exclude, x509)) { > + info("%08lx: excluded", hash); > + X509_free(x509); > + continue; > + } > + if (find_cert(tree, x509)) { > + info("%08lx: duplicate", hash); > + X509_free(x509); > + continue; > + } > + if ((cert =3D calloc(1, sizeof(*cert))) =3D=3D NULL) > + err(1, NULL); > + cert->x509 =3D x509; > + name =3D X509_get_subject_name(x509); > + cert->hash =3D X509_NAME_hash_ex(name, NULL, NULL, NULL); > + cert->name =3D X509_NAME_oneline(name, NULL, 0); > + cert->path =3D xstrdup(unexpand_path(path)); > + if (RB_INSERT(cert_tree, tree, cert) !=3D NULL) > + errx(1, "unexpected duplicate"); > + info("%08lx: %s", cert->hash, strrchr(cert->name, '=3D') + 1); > + no++; > + } > + /* > + * ni is the number of certificates we found in the file. > + * no is the number of certificates that weren't already in our > + * tree or on the exclusion list. > + */ > + if (ni =3D=3D 0) > + warnx("%s: no valid certificates found", path); > + fclose(f); > + return (no); > +} > + > +/* > + * Load all certificates found in the specified path into a tree, > + * optionally excluding those that already exist in a different tree. > + * > + * Returns the number of certificates added to the tree, or -1 on = failure. > + */ > +static int > +read_certs(const char *path, struct cert_tree *tree, struct cert_tree = *exclude) > +{ > + struct stat sb; > + char *paths[] =3D { (char *)(uintptr_t)path, NULL }; > + FTS *fts; > + FTSENT *ent; > + int fts_options =3D FTS_LOGICAL | FTS_NOCHDIR; > + int ret, total =3D 0; > + > + if (stat(path, &sb) !=3D 0) { > + return (-1); > + } else if (!S_ISDIR(sb.st_mode)) { > + errno =3D ENOTDIR; > + return (-1); > + } > + if ((fts =3D fts_open(paths, fts_options, NULL)) =3D=3D NULL) > + err(1, "fts_open()"); > + while ((ent =3D fts_read(fts)) !=3D NULL) { > + if (ent->fts_info !=3D FTS_F) { > + if (ent->fts_info =3D=3D FTS_ERR) > + warnc(ent->fts_errno, "fts_read()"); > + continue; > + } > + info("found %s", ent->fts_path); > + ret =3D read_cert(ent->fts_path, tree, exclude); > + if (ret > 0) > + total +=3D ret; > + } > + fts_close(fts); > + return (total); > +} > + > +/* > + * Save the contents of a cert tree to disk. > + * > + * Returns 0 on success and -1 on failure. > + */ > +static int > +write_certs(const char *dir, struct cert_tree *tree) > +{ > + struct file_tree files =3D RB_INITIALIZER(&files); > + struct cert *cert; > + struct file *file, *tmp; > + struct dirent **dents, **ent; > + char *path, *tmppath =3D NULL; > + FILE *f; > + mode_t mode =3D 0444; > + int cmp, d, fd, ndents, ret =3D 0; > + > + /* > + * Start by generating unambiguous file names for each certificate > + * and storing them in lexicographical order > + */ > + RB_FOREACH(cert, cert_tree, tree) { > + if ((file =3D calloc(1, sizeof(*file))) =3D=3D NULL) > + err(1, NULL); > + file->cert =3D cert; > + for (file->c =3D 0; file->c < INT_MAX; file->c++) > + if (RB_INSERT(file_tree, &files, file) =3D=3D NULL) > + break; > + if (file->c =3D=3D INT_MAX) > + errx(1, "unable to disambiguate %08lx", cert->hash); > + free(cert->path); > + cert->path =3D xasprintf("%08lx.%d", cert->hash, file->c); > + } > + /* > + * Open and scan the directory. > + */ > + if ((d =3D open(dir, O_DIRECTORY | O_RDONLY)) < 0 || > + (ndents =3D fdscandir(d, &dents, NULL, lexisort)) < 0) > + err(1, "%s", dir); > + /* > + * Iterate over the directory listing and the certificate listing > + * in parallel. If the directory listing gets ahead of the > + * certificate listing, we need to write the current certificate > + * and advance the certificate listing. If the certificate > + * listing is ahead of the directory listing, we need to delete > + * the current file and advance the directory listing. If they > + * are neck and neck, we have a match and could in theory compare > + * the two, but in practice it's faster to just replace the > + * current file with the current certificate (and advance both). > + */ > + ent =3D dents; > + file =3D RB_MIN(file_tree, &files); > + for (;;) { > + if (ent < dents + ndents) { > + /* skip directories */ > + if ((*ent)->d_type =3D=3D DT_DIR) { > + free(*ent++); > + continue; > + } > + if (file !=3D NULL) { > + /* compare current dirent to current cert */ > + path =3D file->cert->path; > + cmp =3D strcmp((*ent)->d_name, path); > + } else { > + /* trailing files in directory */ > + path =3D NULL; > + cmp =3D -1; > + } > + } else { > + if (file !=3D NULL) { > + /* trailing certificates */ > + path =3D file->cert->path; > + cmp =3D 1; > + } else { > + /* end of both lists */ > + path =3D NULL; > + break; > + } > + } > + if (cmp < 0) { > + /* a file on disk with no matching certificate */ > + info("removing %s/%s", dir, (*ent)->d_name); > + if (!dryrun) > + (void)unlinkat(d, (*ent)->d_name, 0); > + free(*ent++); > + continue; > + } > + if (cmp =3D=3D 0) { > + /* a file on disk with a matching certificate */ > + info("replacing %s/%s", dir, (*ent)->d_name); > + if (dryrun) { > + fd =3D open(_PATH_DEVNULL, O_WRONLY); > + } else { > + tmppath =3D xasprintf(".%s", path); > + fd =3D openat(d, tmppath, > + O_CREAT | O_WRONLY | O_TRUNC, mode); > + if (!unprivileged && fd >=3D 0) > + (void)fchmod(fd, mode); > + } > + free(*ent++); > + } else { > + /* a certificate with no matching file */ > + info("writing %s/%s", dir, path); > + if (dryrun) { > + fd =3D open(_PATH_DEVNULL, O_WRONLY); > + } else { > + tmppath =3D xasprintf(".%s", path); > + fd =3D openat(d, tmppath, > + O_CREAT | O_WRONLY | O_EXCL, mode); > + } > + } > + /* write the certificate */ > + if (fd < 0 || > + (f =3D fdopen(fd, "w")) =3D=3D NULL || > + !PEM_write_X509(f, file->cert->x509)) { > + if (tmppath !=3D NULL && fd >=3D 0) { > + int serrno =3D errno; > + (void)unlinkat(d, tmppath, 0); > + errno =3D serrno; > + } > + err(1, "%s/%s", dir, tmppath ? tmppath : path); > + } > + /* rename temp file if applicable */ > + if (tmppath !=3D NULL) { > + if (ret =3D=3D 0 && renameat(d, tmppath, d, path) !=3D 0) { > + warn("%s/%s", dir, path); > + ret =3D -1; > + } > + if (ret !=3D 0) > + (void)unlinkat(d, tmppath, 0); > + free(tmppath); > + tmppath =3D NULL; > + } > + /* emit metalog */ > + if (mlf !=3D NULL) { > + fprintf(mlf, "%s/%s type=3Dfile " > + "uname=3D%s gname=3D%s mode=3D%#o size=3D%ld\n", > + unexpand_path(dir), path, > + uname, gname, mode, ftell(f)); > + } > + fclose(f); > + /* advance certificate listing */ > + tmp =3D RB_NEXT(file_tree, &files, file); > + RB_REMOVE(file_tree, &files, file); > + free(file); > + file =3D tmp; > + } > + free(dents); > + close(d); > + return (ret); > +} > + > +/* > + * Save all certs in a tree to a single file (bundle). > + * > + * Returns 0 on success and -1 on failure. > + */ > +static int > +write_bundle(const char *dir, const char *file, struct cert_tree = *tree) > +{ > + struct cert *cert; > + char *tmpfile =3D NULL; > + FILE *f; > + int d, fd, ret =3D 0; > + mode_t mode =3D 0444; > + > + if (dir !=3D NULL) { > + if ((d =3D open(dir, O_DIRECTORY | O_RDONLY)) < 0) > + err(1, "%s", dir); > + } else { > + dir =3D "."; > + d =3D AT_FDCWD; > + } > + info("writing %s/%s", dir, file); > + if (dryrun) { > + fd =3D open(_PATH_DEVNULL, O_WRONLY); > + } else { > + tmpfile =3D xasprintf(".%s", file); > + fd =3D openat(d, tmpfile, O_WRONLY | O_CREAT | O_EXCL, mode); > + } > + if (fd < 0 || (f =3D fdopen(fd, "w")) =3D=3D NULL) { > + if (tmpfile !=3D NULL && fd >=3D 0) { > + int serrno =3D errno; > + (void)unlinkat(d, tmpfile, 0); > + errno =3D serrno; > + } > + err(1, "%s/%s", dir, tmpfile ? tmpfile : file); > + } > + RB_FOREACH(cert, cert_tree, tree) { > + if (!PEM_write_X509(f, cert->x509)) { > + warn("%s/%s", dir, tmpfile ? tmpfile : file); > + ret =3D -1; > + break; > + } > + } > + if (tmpfile !=3D NULL) { > + if (ret =3D=3D 0 && renameat(d, tmpfile, d, file) !=3D 0) { > + warn("%s/%s", dir, file); > + ret =3D -1; > + } > + if (ret !=3D 0) > + (void)unlinkat(d, tmpfile, 0); > + free(tmpfile); > + } > + if (ret =3D=3D 0 && mlf !=3D NULL) { > + fprintf(mlf, > + "%s/%s type=3Dfile uname=3D%s gname=3D%s mode=3D%#o size=3D%ld\n", > + unexpand_path(dir), file, uname, gname, mode, ftell(f)); > + } > + fclose(f); > + if (d !=3D AT_FDCWD) > + close(d); > + return (ret); > +} > + > +/* > + * Load trusted certificates. > + * > + * Returns the number of certificates loaded. > + */ > +static unsigned int > +load_trusted(bool all, struct cert_tree *exclude) > +{ > + unsigned int i, n; > + int ret; > + > + /* load external trusted certs */ > + for (i =3D n =3D 0; all && trusted_paths[i] !=3D NULL; i++) { > + ret =3D read_certs(trusted_paths[i], &trusted, exclude); > + if (ret > 0) > + n +=3D ret; > + } > + > + /* load installed trusted certs */ > + ret =3D read_certs(trusted_dest, &trusted, exclude); > + if (ret > 0) > + n +=3D ret; > + > + info("%d trusted certificates found", n); > + return (n); > +} > + > +/* > + * Load untrusted certificates. > + * > + * Returns the number of certificates loaded. > + */ > +static unsigned int > +load_untrusted(bool all) > +{ > + char *path; > + unsigned int i, n; > + int ret; > + > + /* load external untrusted certs */ > + for (i =3D n =3D 0; all && untrusted_paths[i] !=3D NULL; i++) { > + ret =3D read_certs(untrusted_paths[i], &untrusted, NULL); > + if (ret > 0) > + n +=3D ret; > + } > + > + /* load installed untrusted certs */ > + ret =3D read_certs(untrusted_dest, &untrusted, NULL); > + if (ret > 0) > + n +=3D ret; > + > + /* load legacy untrusted certs */ > + path =3D expand_path(LEGACY_PATH); > + ret =3D read_certs(path, &untrusted, NULL); > + if (ret > 0) { > + warnx("certificates found in legacy directory %s", > + path); > + n +=3D ret; > + } else if (ret =3D=3D 0) { > + warnx("legacy directory %s can safely be deleted", > + path); > + } > + free(path); > + > + info("%d untrusted certificates found", n); > + return (n); > +} > + > +/* > + * Save trusted certificates. > + * > + * Returns 0 on success and -1 on failure. > + */ > +static int > +save_trusted(void) > +{ > + int ret; > + > + /* save untrusted certs */ > + ret =3D write_certs(trusted_dest, &trusted); > + return (ret); > +} > + > +/* > + * Save untrusted certificates. > + * > + * Returns 0 on success and -1 on failure. > + */ > +static int > +save_untrusted(void) > +{ > + int ret; > + > + ret =3D write_certs(untrusted_dest, &untrusted); > + return (ret); > +} > + > +/* > + * Save certificate bundle. > + * > + * Returns 0 on success and -1 on failure. > + */ > +static int > +save_bundle(void) > +{ > + char *dir, *file, *sep; > + int ret; > + > + if ((sep =3D strrchr(bundle_dest, '/')) =3D=3D NULL) { > + dir =3D NULL; > + file =3D bundle_dest; > + } else { > + dir =3D xasprintf("%.*s", (int)(sep - bundle_dest), bundle_dest); > + file =3D sep + 1; > + } > + ret =3D write_bundle(dir, file, &trusted); > + free(dir); > + return (ret); > +} > + > +/* > *** 1032 LINES SKIPPED ***