From nobody Wed Aug 17 21:29:01 2022 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4M7Lk55XG8z4ZKJr; Wed, 17 Aug 2022 21:29:01 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4M7Lk54wbHz3Mbt; Wed, 17 Aug 2022 21:29:01 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1660771741; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=rzwH1Jmaiv034jML24+CIhHSEgf43kdxTegB0TT19Is=; b=JWRzJPlJl/eEzaKpoeJPXZTLcckQ6lei6Qt6xXomi3O8P4YK2nK8g9xs1EofupwdLHnoXV 0GwaqnGEiZWglPQHPpAICxgL1kSUxlAtLi3K3i2j5mHBnJOwn5vjRNNemvtcIP8n2TjgR8 VwP6JbbJx9DWLWbasIDqoU66BA3VFo8Y0GECc8Hn5A0coJDEz/P7EAuJy/gjt/nVhhdsE2 42XhYnHMI59MWbhHyvpK1dKiSizoqsyjlN7/TSzaSlDe63mDdmsNIBUc4RYSnWYE5muCKt 754qSKle+DKSEOvDubZeSGH3UYhGfD68bX15IwCqUIaA+KJUmqOn/x6oOzui2Q== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4M7Lk53yvXz14HQ; Wed, 17 Aug 2022 21:29:01 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 27HLT1Qp028005; Wed, 17 Aug 2022 21:29:01 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 27HLT1Ng028004; Wed, 17 Aug 2022 21:29:01 GMT (envelope-from git) Date: Wed, 17 Aug 2022 21:29:01 GMT Message-Id: <202208172129.27HLT1Ng028004@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Mark Johnston Subject: git: 8eca3207980a - main - makefs: Handle multiple staging directories when creating ZFS pools List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-all@freebsd.org X-BeenThere: dev-commits-src-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: markj X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 8eca3207980a8c2f6457c1cd2e9ff6b235a3018d Auto-Submitted: auto-generated ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1660771741; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=rzwH1Jmaiv034jML24+CIhHSEgf43kdxTegB0TT19Is=; b=Al4hOKd/T9yoDbSIW3EgRUO/AeRKQYA69vkenX336qlwuXkMlMYuFpfCwwX9fz50/8rENB w0HuqtnVVtnBCTwVzVGImOXMmil1zCq3YiiSUmcJBTNnreYvRb8xzmdlCRGXKFiRdguWWL z1C0Tt2pSjnaEhxEMM8XAHI++7PZKIB+XaKeCA8OvtS7z2tsvFFtNlMVWyu3i+6WEs0J7b FEnbK0G0HScXuUCW2XSrRf2HREPBYcVNBqr6ZAcASrbxlE6PFBGum2hv4e92BwyfmfRtZn LEWqILH71CTMHxTUAm84k/IE4YH2lG2+FZH+ynuNHZrPvDzsqJQpqSfYOM0tqA== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1660771741; a=rsa-sha256; cv=none; b=UopTiuwF08xJDGD9hw0ykeHSB9V2+x+WPqj+3TNcRcb3vJOD7iwUN9NDIrXKGP7MzJs52B +M7DI/QWKYXo8h37yRgBK9/ubAkKEXdc1M0cbw4l+RKDxSwlxZzKoMUhu3ucy0n5QCRY1v yjV0qHtSAXtchMtzqBy3Po7bRzSLsenO8z/45lNKU60S5E1ZRbgOVELgioju+XM6UwhcID 7spIXajJxfdbqcKIGQkvisZOrha/M58wOPpiqso5Ok4536lKG1sp/MPTPr9nsCd78LGmGc ck35wKBtEg7J2szYWIRrxfG3h9bTF0GG8JMGc2/A8T4SRgD4vfAnYiYo6nH65Q== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N The branch main has been updated by markj: URL: https://cgit.FreeBSD.org/src/commit/?id=8eca3207980a8c2f6457c1cd2e9ff6b235a3018d commit 8eca3207980a8c2f6457c1cd2e9ff6b235a3018d Author: Mark Johnston AuthorDate: 2022-08-17 19:56:57 +0000 Commit: Mark Johnston CommitDate: 2022-08-17 21:28:00 +0000 makefs: Handle multiple staging directories when creating ZFS pools The fsnode tree traversal routines used in ZFS mode assume that all children of a (directory) fsnode can be accessed using a directory fd for the parent and the child name. This is true when populating the image using an mtree manifest or from a single staging directory, but doesn't work when multiple staging directories are specified. Change the traversal routines to use absolute path lookups when an mtree manifest is not in use. This isn't ideal, but it's the simplest way to fix the problem. Reported by: imp Sponsored by: The FreeBSD Foundation --- usr.sbin/makefs/zfs/fs.c | 99 +++++++++++++++++++++++++++++++++++++----------- 1 file changed, 77 insertions(+), 22 deletions(-) diff --git a/usr.sbin/makefs/zfs/fs.c b/usr.sbin/makefs/zfs/fs.c index 2907a6b05f81..b246a8e23f4e 100644 --- a/usr.sbin/makefs/zfs/fs.c +++ b/usr.sbin/makefs/zfs/fs.c @@ -156,13 +156,20 @@ struct fs_populate_dir { struct fs_populate_arg { zfs_opt_t *zfs; zfs_fs_t *fs; /* owning filesystem */ - int dirfd; /* current directory fd */ uint64_t rootdirid; /* root directory dnode ID */ + int rootdirfd; /* root directory fd */ SLIST_HEAD(, fs_populate_dir) dirs; /* stack of directories */ }; static void fs_build_one(zfs_opt_t *, zfs_dsl_dir_t *, fsnode *, int); +static void +eclose(int fd) +{ + if (close(fd) != 0) + err(1, "close"); +} + static bool fsnode_isroot(const fsnode *cur) { @@ -235,6 +242,65 @@ fs_populate_varszattr(zfs_fs_t *fs, char *attrbuf, const void *val, *szp += valsz; } +/* + * Derive the relative fd/path combo needed to access a file. Ideally we'd + * always be able to use relative lookups (i.e., use the *at() system calls), + * since they require less path translation and are more amenable to sandboxing, + * but the handling of multiple staging directories makes that difficult. To + * make matters worse, we have no choice but to use relative lookups when + * dealing with an mtree manifest, so both mechanisms are implemented. + */ +static void +fs_populate_path(const fsnode *cur, struct fs_populate_arg *arg, + char *path, size_t sz, int *dirfdp) +{ + if (cur->root == NULL) { + size_t n; + + *dirfdp = SLIST_FIRST(&arg->dirs)->dirfd; + n = strlcpy(path, cur->name, sz); + assert(n < sz); + } else { + int n; + + *dirfdp = AT_FDCWD; + n = snprintf(path, sz, "%s/%s/%s", + cur->root, cur->path, cur->name); + assert(n >= 0); + assert((size_t)n < sz); + } +} + +static int +fs_open(const fsnode *cur, struct fs_populate_arg *arg, int flags) +{ + char path[PATH_MAX]; + int fd; + + fs_populate_path(cur, arg, path, sizeof(path), &fd); + + fd = openat(fd, path, flags); + if (fd < 0) + err(1, "openat(%s)", path); + return (fd); +} + +static void +fs_readlink(const fsnode *cur, struct fs_populate_arg *arg, + char *buf, size_t bufsz) +{ + char path[PATH_MAX]; + ssize_t n; + int fd; + + fs_populate_path(cur, arg, path, sizeof(path), &fd); + + n = readlinkat(fd, path, buf, bufsz - 1); + if (n == -1) + err(1, "readlinkat(%s)", cur->name); + buf[n] = '\0'; +} + static void fs_populate_sattrs(struct fs_populate_arg *arg, const fsnode *cur, dnode_phys_t *dnode) @@ -283,20 +349,14 @@ fs_populate_sattrs(struct fs_populate_arg *arg, const fsnode *cur, parent = SLIST_EMPTY(&arg->dirs) ? arg->rootdirid : SLIST_FIRST(&arg->dirs)->objid; break; - case S_IFLNK: { - ssize_t n; - - if ((n = readlinkat(SLIST_FIRST(&arg->dirs)->dirfd, cur->name, - target, sizeof(target) - 1)) == -1) - err(1, "readlinkat(%s)", cur->name); - target[n] = '\0'; + case S_IFLNK: + fs_readlink(cur, arg, target, sizeof(target)); layout = SA_LAYOUT_INDEX_SYMLINK; links = 1; objsize = strlen(target); parent = SLIST_FIRST(&arg->dirs)->objid; break; - } default: assert(0); } @@ -438,9 +498,7 @@ fs_populate_file(fsnode *cur, struct fs_populate_arg *arg) cur->inode->ino = dnid; cur->inode->flags |= FI_ALLOCATED; - fd = openat(SLIST_FIRST(&arg->dirs)->dirfd, cur->name, O_RDONLY); - if (fd == -1) - err(1, "openat(%s)", cur->name); + fd = fs_open(cur, arg, O_RDONLY); buf = zfs->filebuf; bufsz = sizeof(zfs->filebuf); @@ -473,8 +531,7 @@ fs_populate_file(fsnode *cur, struct fs_populate_arg *arg) vdev_pwrite_dnode_indir(zfs, dnode, 0, 1, buf, target, loc, dnode_cursor_next(zfs, c, foff)); } - if (close(fd) != 0) - err(1, "close"); + eclose(fd); dnode_cursor_finish(zfs, c); fs_populate_sattrs(arg, cur, dnode); @@ -502,13 +559,11 @@ fs_populate_dir(fsnode *cur, struct fs_populate_arg *arg) */ if (!SLIST_EMPTY(&arg->dirs)) { fs_populate_dirent(arg, cur, dnid); - dirfd = openat(SLIST_FIRST(&arg->dirs)->dirfd, cur->name, - O_DIRECTORY); - if (dirfd < 0) - err(1, "open(%s)", cur->name); + dirfd = fs_open(cur, arg, O_DIRECTORY | O_RDONLY); } else { arg->rootdirid = dnid; - dirfd = arg->dirfd; + dirfd = arg->rootdirfd; + arg->rootdirfd = -1; } /* @@ -589,8 +644,8 @@ fs_foreach_populate(fsnode *cur, void *_arg) dir = SLIST_FIRST(&arg->dirs); SLIST_REMOVE_HEAD(&arg->dirs, next); zap_write(arg->zfs, dir->zap); - if (dir->dirfd != -1 && close(dir->dirfd) != 0) - err(1, "close"); + if (dir->dirfd != -1) + eclose(dir->dirfd); free(dir); cur = cur->parent; } while (cur != NULL && cur->next == NULL && @@ -881,7 +936,7 @@ fs_build_one(zfs_opt_t *zfs, zfs_dsl_dir_t *dsldir, fsnode *root, int dirfd) * Make a second pass to populate the dataset with files from the * staged directory. Most of our runtime is spent here. */ - arg.dirfd = dirfd; + arg.rootdirfd = dirfd; arg.zfs = zfs; arg.fs = &fs; SLIST_INIT(&arg.dirs);