svn commit: r323530 - vendor-sys/illumos/dist/common/zfs vendor-sys/illumos/dist/uts/common vendor-sys/illumos/dist/uts/common/fs/zfs vendor-sys/illumos/dist/uts/common/fs/zfs/lua vendor-sys/illumo...
Andriy Gapon
avg at FreeBSD.org
Wed Sep 13 10:45:52 UTC 2017
Author: avg
Date: Wed Sep 13 10:45:49 2017
New Revision: 323530
URL: https://svnweb.freebsd.org/changeset/base/323530
Log:
7431 ZFS Channel Programs
illumos/illumos-gate at dfc115332c94a2f62058ac7f2bce7631fbd20b3d
https://github.com/illumos/illumos-gate/commit/dfc115332c94a2f62058ac7f2bce7631fbd20b3d
https://www.illumos.org/issues/7431
ZFS channel programs (ZCP) adds support for performing compound ZFS
administrative actions via Lua scripts in a sandboxed environment (with time
and memory limits).
This initial commit includes both base support for running ZCP scripts, and a
small initial library of API calls which support getting properties and
listing, destroying, and promoting datasets.
Testing: in addition to the included unit tests, channel programs have been in
use at Delphix for several months for batch destroying filesystems. The
dsl_destroy_snaps_nvl() call has also been replaced with
For reference, the new zfs-program manpage is included below.
ZFS-PROGRAM(1M) 1M ZFS-PROGRAM(1M)
NAME
zfs program – executes ZFS channel programs
SYNOPSIS
zfs program [-t timeout] [-m memory-limit] pool script
DESCRIPTION
The ZFS channel program interface allows ZFS administrative operations to
be run programmatically as a Lua script. The entire script is executed
atomically, with no other administrative operations taking effect
concurrently. A library of ZFS calls is made available to channel program
scripts. Channel programs may only be run with root privileges.
A modified version of the Lua 5.2 interpreter is used to run channel
program scripts. The Lua 5.2 manual can be found at:
http://www.lua.org/manual/5.2/
...
Reviewed by: Matthew Ahrens <mahrens at delphix.com>
Reviewed by: George Wilson <george.wilson at delphix.com>
Reviewed by: John Kennedy <john.kennedy at delphix.com>
Reviewed by: Dan Kimmel <dan.kimmel at delphix.com>
Approved by: Garrett D'Amore <garrett at damore.org>
Author: Chris Williamson <chris.williamson at delphix.com>
Added:
vendor/illumos/dist/man/man1m/zfs-program.1m
Modified:
vendor/illumos/dist/cmd/zfs/zfs_main.c
vendor/illumos/dist/cmd/zpool/zpool_main.c
vendor/illumos/dist/lib/libzfs/common/libzfs_dataset.c
vendor/illumos/dist/lib/libzfs/common/libzfs_impl.h
vendor/illumos/dist/lib/libzfs/common/libzfs_util.c
vendor/illumos/dist/lib/libzfs_core/common/libzfs_core.c
vendor/illumos/dist/lib/libzfs_core/common/libzfs_core.h
vendor/illumos/dist/lib/libzpool/common/kernel.c
vendor/illumos/dist/lib/libzpool/common/sys/zfs_context.h
vendor/illumos/dist/man/man1m/zfs.1m
Changes in other areas also in this revision:
Added:
vendor-sys/illumos/dist/uts/common/fs/zfs/lua/
vendor-sys/illumos/dist/uts/common/fs/zfs/lua/README.zfs
vendor-sys/illumos/dist/uts/common/fs/zfs/lua/lapi.c (contents, props changed)
vendor-sys/illumos/dist/uts/common/fs/zfs/lua/lapi.h (contents, props changed)
vendor-sys/illumos/dist/uts/common/fs/zfs/lua/lauxlib.c (contents, props changed)
vendor-sys/illumos/dist/uts/common/fs/zfs/lua/lauxlib.h (contents, props changed)
vendor-sys/illumos/dist/uts/common/fs/zfs/lua/lbaselib.c (contents, props changed)
vendor-sys/illumos/dist/uts/common/fs/zfs/lua/lbitlib.c (contents, props changed)
vendor-sys/illumos/dist/uts/common/fs/zfs/lua/lcode.c (contents, props changed)
vendor-sys/illumos/dist/uts/common/fs/zfs/lua/lcode.h (contents, props changed)
vendor-sys/illumos/dist/uts/common/fs/zfs/lua/lcompat.c (contents, props changed)
vendor-sys/illumos/dist/uts/common/fs/zfs/lua/lcorolib.c (contents, props changed)
vendor-sys/illumos/dist/uts/common/fs/zfs/lua/lctype.c (contents, props changed)
vendor-sys/illumos/dist/uts/common/fs/zfs/lua/lctype.h (contents, props changed)
vendor-sys/illumos/dist/uts/common/fs/zfs/lua/ldebug.c (contents, props changed)
vendor-sys/illumos/dist/uts/common/fs/zfs/lua/ldebug.h (contents, props changed)
vendor-sys/illumos/dist/uts/common/fs/zfs/lua/ldo.c (contents, props changed)
vendor-sys/illumos/dist/uts/common/fs/zfs/lua/ldo.h (contents, props changed)
vendor-sys/illumos/dist/uts/common/fs/zfs/lua/ldump.c (contents, props changed)
vendor-sys/illumos/dist/uts/common/fs/zfs/lua/lfunc.c (contents, props changed)
vendor-sys/illumos/dist/uts/common/fs/zfs/lua/lfunc.h (contents, props changed)
vendor-sys/illumos/dist/uts/common/fs/zfs/lua/lgc.c (contents, props changed)
vendor-sys/illumos/dist/uts/common/fs/zfs/lua/lgc.h (contents, props changed)
vendor-sys/illumos/dist/uts/common/fs/zfs/lua/llex.c (contents, props changed)
vendor-sys/illumos/dist/uts/common/fs/zfs/lua/llex.h (contents, props changed)
vendor-sys/illumos/dist/uts/common/fs/zfs/lua/llimits.h (contents, props changed)
vendor-sys/illumos/dist/uts/common/fs/zfs/lua/lmem.c (contents, props changed)
vendor-sys/illumos/dist/uts/common/fs/zfs/lua/lmem.h (contents, props changed)
vendor-sys/illumos/dist/uts/common/fs/zfs/lua/lobject.c (contents, props changed)
vendor-sys/illumos/dist/uts/common/fs/zfs/lua/lobject.h (contents, props changed)
vendor-sys/illumos/dist/uts/common/fs/zfs/lua/lopcodes.c (contents, props changed)
vendor-sys/illumos/dist/uts/common/fs/zfs/lua/lopcodes.h (contents, props changed)
vendor-sys/illumos/dist/uts/common/fs/zfs/lua/lparser.c (contents, props changed)
vendor-sys/illumos/dist/uts/common/fs/zfs/lua/lparser.h (contents, props changed)
vendor-sys/illumos/dist/uts/common/fs/zfs/lua/lstate.c (contents, props changed)
vendor-sys/illumos/dist/uts/common/fs/zfs/lua/lstate.h (contents, props changed)
vendor-sys/illumos/dist/uts/common/fs/zfs/lua/lstring.c (contents, props changed)
vendor-sys/illumos/dist/uts/common/fs/zfs/lua/lstring.h (contents, props changed)
vendor-sys/illumos/dist/uts/common/fs/zfs/lua/lstrlib.c (contents, props changed)
vendor-sys/illumos/dist/uts/common/fs/zfs/lua/ltable.c (contents, props changed)
vendor-sys/illumos/dist/uts/common/fs/zfs/lua/ltable.h (contents, props changed)
vendor-sys/illumos/dist/uts/common/fs/zfs/lua/ltablib.c (contents, props changed)
vendor-sys/illumos/dist/uts/common/fs/zfs/lua/ltm.c (contents, props changed)
vendor-sys/illumos/dist/uts/common/fs/zfs/lua/ltm.h (contents, props changed)
vendor-sys/illumos/dist/uts/common/fs/zfs/lua/lua.h (contents, props changed)
vendor-sys/illumos/dist/uts/common/fs/zfs/lua/luaconf.h (contents, props changed)
vendor-sys/illumos/dist/uts/common/fs/zfs/lua/lualib.h (contents, props changed)
vendor-sys/illumos/dist/uts/common/fs/zfs/lua/lundump.c (contents, props changed)
vendor-sys/illumos/dist/uts/common/fs/zfs/lua/lundump.h (contents, props changed)
vendor-sys/illumos/dist/uts/common/fs/zfs/lua/lvm.c (contents, props changed)
vendor-sys/illumos/dist/uts/common/fs/zfs/lua/lvm.h (contents, props changed)
vendor-sys/illumos/dist/uts/common/fs/zfs/lua/lzio.c (contents, props changed)
vendor-sys/illumos/dist/uts/common/fs/zfs/lua/lzio.h (contents, props changed)
vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zcp.h (contents, props changed)
vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zcp_global.h (contents, props changed)
vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zcp_iter.h (contents, props changed)
vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zcp_prop.h (contents, props changed)
vendor-sys/illumos/dist/uts/common/fs/zfs/zcp.c (contents, props changed)
vendor-sys/illumos/dist/uts/common/fs/zfs/zcp_get.c (contents, props changed)
vendor-sys/illumos/dist/uts/common/fs/zfs/zcp_global.c (contents, props changed)
vendor-sys/illumos/dist/uts/common/fs/zfs/zcp_iter.c (contents, props changed)
vendor-sys/illumos/dist/uts/common/fs/zfs/zcp_synctask.c (contents, props changed)
Modified:
vendor-sys/illumos/dist/common/zfs/zfs_prop.c
vendor-sys/illumos/dist/uts/common/Makefile.files
vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_dataset.c
vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_destroy.c
vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_dir.c
vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dsl_dataset.h
vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dsl_destroy.h
vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dsl_dir.h
vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zfs_ioctl.h
vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zfs_vfsops.h
vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_ioctl.c
vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_vfsops.c
vendor-sys/illumos/dist/uts/common/sys/fs/zfs.h
Modified: vendor/illumos/dist/cmd/zfs/zfs_main.c
==============================================================================
--- vendor/illumos/dist/cmd/zfs/zfs_main.c Wed Sep 13 10:41:47 2017 (r323529)
+++ vendor/illumos/dist/cmd/zfs/zfs_main.c Wed Sep 13 10:45:49 2017 (r323530)
@@ -21,7 +21,7 @@
/*
* Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved.
- * Copyright (c) 2011, 2015 by Delphix. All rights reserved.
+ * Copyright (c) 2011, 2016 by Delphix. All rights reserved.
* Copyright 2012 Milan Jurik. All rights reserved.
* Copyright (c) 2012, Joyent, Inc. All rights reserved.
* Copyright (c) 2011-2012 Pawel Jakub Dawidek. All rights reserved.
@@ -50,6 +50,7 @@
#include <grp.h>
#include <pwd.h>
#include <signal.h>
+#include <sys/debug.h>
#include <sys/list.h>
#include <sys/mkdev.h>
#include <sys/mntent.h>
@@ -106,6 +107,7 @@ static int zfs_do_holds(int argc, char **argv);
static int zfs_do_release(int argc, char **argv);
static int zfs_do_diff(int argc, char **argv);
static int zfs_do_bookmark(int argc, char **argv);
+static int zfs_do_channel_program(int argc, char **argv);
/*
* Enable a reasonable set of defaults for libumem debugging on DEBUG builds.
@@ -153,6 +155,7 @@ typedef enum {
HELP_RELEASE,
HELP_DIFF,
HELP_BOOKMARK,
+ HELP_CHANNEL_PROGRAM,
} zfs_help_t;
typedef struct zfs_command {
@@ -180,6 +183,7 @@ static zfs_command_t command_table[] = {
{ "promote", zfs_do_promote, HELP_PROMOTE },
{ "rename", zfs_do_rename, HELP_RENAME },
{ "bookmark", zfs_do_bookmark, HELP_BOOKMARK },
+ { "program", zfs_do_channel_program, HELP_CHANNEL_PROGRAM },
{ NULL },
{ "list", zfs_do_list, HELP_LIST },
{ NULL },
@@ -324,6 +328,10 @@ get_usage(zfs_help_t idx)
"[snapshot|filesystem]\n"));
case HELP_BOOKMARK:
return (gettext("\tbookmark <snapshot> <bookmark>\n"));
+ case HELP_CHANNEL_PROGRAM:
+ return (gettext("\tprogram [-t <instruction limit>] "
+ "[-m <memory limit (b)>] <pool> <program file> "
+ "[lua args...]\n"));
}
abort();
@@ -352,6 +360,18 @@ safe_malloc(size_t size)
return (data);
}
+void *
+safe_realloc(void *data, size_t size)
+{
+ void *newp;
+ if ((newp = realloc(data, size)) == NULL) {
+ free(data);
+ nomem();
+ }
+
+ return (newp);
+}
+
static char *
safe_strdup(char *str)
{
@@ -6945,6 +6965,190 @@ zfs_do_bookmark(int argc, char **argv)
dgettext(TEXT_DOMAIN, err_msg));
}
+ return (ret != 0);
+
+usage:
+ usage(B_FALSE);
+ return (-1);
+}
+
+static int
+zfs_do_channel_program(int argc, char **argv)
+{
+ int ret, fd;
+ char c;
+ char *progbuf, *filename, *poolname;
+ size_t progsize, progread;
+ nvlist_t *outnvl;
+ uint64_t instrlimit = ZCP_DEFAULT_INSTRLIMIT;
+ uint64_t memlimit = ZCP_DEFAULT_MEMLIMIT;
+ zpool_handle_t *zhp;
+
+ /* check options */
+ while (-1 !=
+ (c = getopt(argc, argv, "t:(instr-limit)m:(memory-limit)"))) {
+ switch (c) {
+ case 't':
+ case 'm': {
+ uint64_t arg;
+ char *endp;
+
+ errno = 0;
+ arg = strtoull(optarg, &endp, 0);
+ if (errno != 0 || *endp != '\0') {
+ (void) fprintf(stderr, gettext(
+ "invalid argument "
+ "'%s': expected integer\n"), optarg);
+ goto usage;
+ }
+
+ if (c == 't') {
+ if (arg > ZCP_MAX_INSTRLIMIT || arg == 0) {
+ (void) fprintf(stderr, gettext(
+ "Invalid instruction limit: "
+ "%s\n"), optarg);
+ return (1);
+ } else {
+ instrlimit = arg;
+ }
+ } else {
+ ASSERT3U(c, ==, 'm');
+ if (arg > ZCP_MAX_MEMLIMIT || arg == 0) {
+ (void) fprintf(stderr, gettext(
+ "Invalid memory limit: "
+ "%s\n"), optarg);
+ return (1);
+ } else {
+ memlimit = arg;
+ }
+ }
+ break;
+ }
+ case '?':
+ (void) fprintf(stderr, gettext("invalid option '%c'\n"),
+ optopt);
+ goto usage;
+ }
+ }
+
+ argc -= optind;
+ argv += optind;
+
+ if (argc < 2) {
+ (void) fprintf(stderr,
+ gettext("invalid number of arguments\n"));
+ goto usage;
+ }
+
+ poolname = argv[0];
+ filename = argv[1];
+ if (strcmp(filename, "-") == 0) {
+ fd = 0;
+ filename = "standard input";
+ } else if ((fd = open(filename, O_RDONLY)) < 0) {
+ (void) fprintf(stderr, gettext("cannot open '%s': %s\n"),
+ filename, strerror(errno));
+ return (1);
+ }
+
+ if ((zhp = zpool_open(g_zfs, poolname)) == NULL) {
+ (void) fprintf(stderr, gettext("cannot open pool '%s'"),
+ poolname);
+ return (1);
+ }
+ zpool_close(zhp);
+
+ /*
+ * Read in the channel program, expanding the program buffer as
+ * necessary.
+ */
+ progread = 0;
+ progsize = 1024;
+ progbuf = safe_malloc(progsize);
+ do {
+ ret = read(fd, progbuf + progread, progsize - progread);
+ progread += ret;
+ if (progread == progsize && ret > 0) {
+ progsize *= 2;
+ progbuf = safe_realloc(progbuf, progsize);
+ }
+ } while (ret > 0);
+
+ if (fd != 0)
+ (void) close(fd);
+ if (ret < 0) {
+ free(progbuf);
+ (void) fprintf(stderr,
+ gettext("cannot read '%s': %s\n"),
+ filename, strerror(errno));
+ return (1);
+ }
+ progbuf[progread] = '\0';
+
+ /*
+ * Any remaining arguments are passed as arguments to the lua script as
+ * a string array:
+ * {
+ * "argv" -> [ "arg 1", ... "arg n" ],
+ * }
+ */
+ nvlist_t *argnvl = fnvlist_alloc();
+ fnvlist_add_string_array(argnvl, ZCP_ARG_CLIARGV, argv + 2, argc - 2);
+
+ ret = lzc_channel_program(poolname, progbuf, instrlimit, memlimit,
+ argnvl, &outnvl);
+
+ if (ret != 0) {
+ /*
+ * On error, report the error message handed back by lua if one
+ * exists. Otherwise, generate an appropriate error message,
+ * falling back on strerror() for an unexpected return code.
+ */
+ char *errstring = NULL;
+ if (nvlist_exists(outnvl, ZCP_RET_ERROR)) {
+ (void) nvlist_lookup_string(outnvl,
+ ZCP_RET_ERROR, &errstring);
+ if (errstring == NULL)
+ errstring = strerror(ret);
+ } else {
+ switch (ret) {
+ case EINVAL:
+ errstring =
+ "Invalid instruction or memory limit.";
+ break;
+ case ENOMEM:
+ errstring = "Return value too large.";
+ break;
+ case ENOSPC:
+ errstring = "Memory limit exhausted.";
+ break;
+ case ETIME:
+ errstring = "Timed out.";
+ break;
+ case EPERM:
+ errstring = "Permission denied. Channel "
+ "programs must be run as root.";
+ break;
+ default:
+ errstring = strerror(ret);
+ }
+ }
+ (void) fprintf(stderr,
+ gettext("Channel program execution failed:\n%s\n"),
+ errstring);
+ } else {
+ (void) printf("Channel program fully executed ");
+ if (nvlist_empty(outnvl)) {
+ (void) printf("with no return value.\n");
+ } else {
+ (void) printf("with return value:\n");
+ dump_nvlist(outnvl, 4);
+ }
+ }
+
+ free(progbuf);
+ fnvlist_free(outnvl);
+ fnvlist_free(argnvl);
return (ret != 0);
usage:
Modified: vendor/illumos/dist/cmd/zpool/zpool_main.c
==============================================================================
--- vendor/illumos/dist/cmd/zpool/zpool_main.c Wed Sep 13 10:41:47 2017 (r323529)
+++ vendor/illumos/dist/cmd/zpool/zpool_main.c Wed Sep 13 10:45:49 2017 (r323530)
@@ -21,7 +21,7 @@
/*
* Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved.
- * Copyright (c) 2011, 2015 by Delphix. All rights reserved.
+ * Copyright (c) 2011, 2016 by Delphix. All rights reserved.
* Copyright (c) 2012 by Frederik Wessels. All rights reserved.
* Copyright (c) 2013 by Prasad Joshi (sTec). All rights reserved.
* Copyright 2016 Igor Kozhukhov <ikozhukhov at gmail.com>.
@@ -5159,6 +5159,11 @@ get_history_one(zpool_handle_t *zhp, void *data)
(void) printf(" output:\n");
dump_nvlist(fnvlist_lookup_nvlist(rec,
ZPOOL_HIST_OUTPUT_NVL), 8);
+ }
+ if (nvlist_exists(rec, ZPOOL_HIST_ERRNO)) {
+ (void) printf(" errno: %lld\n",
+ fnvlist_lookup_int64(rec,
+ ZPOOL_HIST_ERRNO));
}
} else {
if (!cb->internal)
Modified: vendor/illumos/dist/lib/libzfs/common/libzfs_dataset.c
==============================================================================
--- vendor/illumos/dist/lib/libzfs/common/libzfs_dataset.c Wed Sep 13 10:41:47 2017 (r323529)
+++ vendor/illumos/dist/lib/libzfs/common/libzfs_dataset.c Wed Sep 13 10:45:49 2017 (r323530)
@@ -2323,6 +2323,74 @@ zfs_get_clones_nvl(zfs_handle_t *zhp)
}
/*
+ * Accepts a property and value and checks that the value
+ * matches the one found by the channel program. If they are
+ * not equal, print both of them.
+ */
+void
+zcp_check(zfs_handle_t *zhp, zfs_prop_t prop, uint64_t intval,
+ const char *strval)
+{
+ if (!zhp->zfs_hdl->libzfs_prop_debug)
+ return;
+ int error;
+ char *poolname = zhp->zpool_hdl->zpool_name;
+ const char *program =
+ "args = ...\n"
+ "ds = args['dataset']\n"
+ "prop = args['property']\n"
+ "value, setpoint = zfs.get_prop(ds, prop)\n"
+ "return {value=value, setpoint=setpoint}\n";
+ nvlist_t *outnvl;
+ nvlist_t *retnvl;
+ nvlist_t *argnvl = fnvlist_alloc();
+
+ fnvlist_add_string(argnvl, "dataset", zhp->zfs_name);
+ fnvlist_add_string(argnvl, "property", zfs_prop_to_name(prop));
+
+ error = lzc_channel_program(poolname, program,
+ 10 * 1000 * 1000, 10 * 1024 * 1024, argnvl, &outnvl);
+
+ if (error == 0) {
+ retnvl = fnvlist_lookup_nvlist(outnvl, "return");
+ if (zfs_prop_get_type(prop) == PROP_TYPE_NUMBER) {
+ int64_t ans;
+ error = nvlist_lookup_int64(retnvl, "value", &ans);
+ if (error != 0) {
+ (void) fprintf(stderr, "zcp check error: %u\n",
+ error);
+ return;
+ }
+ if (ans != intval) {
+ (void) fprintf(stderr,
+ "%s: zfs found %lld, but zcp found %lld\n",
+ zfs_prop_to_name(prop),
+ (longlong_t)intval, (longlong_t)ans);
+ }
+ } else {
+ char *str_ans;
+ error = nvlist_lookup_string(retnvl, "value", &str_ans);
+ if (error != 0) {
+ (void) fprintf(stderr, "zcp check error: %u\n",
+ error);
+ return;
+ }
+ if (strcmp(strval, str_ans) != 0) {
+ (void) fprintf(stderr,
+ "%s: zfs found %s, but zcp found %s\n",
+ zfs_prop_to_name(prop),
+ strval, str_ans);
+ }
+ }
+ } else {
+ (void) fprintf(stderr,
+ "zcp check failed, channel program error: %u\n", error);
+ }
+ nvlist_free(argnvl);
+ nvlist_free(outnvl);
+}
+
+/*
* Retrieve a property from the given object. If 'literal' is specified, then
* numbers are left as exact values. Otherwise, numbers are converted to a
* human-readable form.
@@ -2368,6 +2436,7 @@ zfs_prop_get(zfs_handle_t *zhp, zfs_prop_t prop, char
&t) == 0)
(void) snprintf(propbuf, proplen, "%llu", val);
}
+ zcp_check(zhp, prop, val, NULL);
break;
case ZFS_PROP_MOUNTPOINT:
@@ -2436,7 +2505,7 @@ zfs_prop_get(zfs_handle_t *zhp, zfs_prop_t prop, char
/* 'legacy' or 'none' */
(void) strlcpy(propbuf, str, proplen);
}
-
+ zcp_check(zhp, prop, NULL, propbuf);
break;
case ZFS_PROP_ORIGIN:
@@ -2444,6 +2513,7 @@ zfs_prop_get(zfs_handle_t *zhp, zfs_prop_t prop, char
if (str == NULL)
return (-1);
(void) strlcpy(propbuf, str, proplen);
+ zcp_check(zhp, prop, NULL, str);
break;
case ZFS_PROP_CLONES:
@@ -2458,7 +2528,6 @@ zfs_prop_get(zfs_handle_t *zhp, zfs_prop_t prop, char
if (get_numeric_property(zhp, prop, src, &source, &val) != 0)
return (-1);
-
/*
* If quota or reservation is 0, we translate this into 'none'
* (unless literal is set), and indicate that it's the default
@@ -2477,6 +2546,7 @@ zfs_prop_get(zfs_handle_t *zhp, zfs_prop_t prop, char
else
zfs_nicenum(val, propbuf, proplen);
}
+ zcp_check(zhp, prop, val, NULL);
break;
case ZFS_PROP_FILESYSTEM_LIMIT:
@@ -2501,6 +2571,8 @@ zfs_prop_get(zfs_handle_t *zhp, zfs_prop_t prop, char
} else {
zfs_nicenum(val, propbuf, proplen);
}
+
+ zcp_check(zhp, prop, val, NULL);
break;
case ZFS_PROP_REFRATIO:
@@ -2510,6 +2582,7 @@ zfs_prop_get(zfs_handle_t *zhp, zfs_prop_t prop, char
(void) snprintf(propbuf, proplen, "%llu.%02llux",
(u_longlong_t)(val / 100),
(u_longlong_t)(val % 100));
+ zcp_check(zhp, prop, val, NULL);
break;
case ZFS_PROP_TYPE:
@@ -2530,6 +2603,7 @@ zfs_prop_get(zfs_handle_t *zhp, zfs_prop_t prop, char
abort();
}
(void) snprintf(propbuf, proplen, "%s", str);
+ zcp_check(zhp, prop, NULL, propbuf);
break;
case ZFS_PROP_MOUNTED:
@@ -2555,6 +2629,7 @@ zfs_prop_get(zfs_handle_t *zhp, zfs_prop_t prop, char
* consumers.
*/
(void) strlcpy(propbuf, zhp->zfs_name, proplen);
+ zcp_check(zhp, prop, NULL, propbuf);
break;
case ZFS_PROP_MLSLABEL:
@@ -2604,26 +2679,33 @@ zfs_prop_get(zfs_handle_t *zhp, zfs_prop_t prop, char
if (get_numeric_property(zhp, prop, src, &source, &val) != 0)
return (-1);
(void) snprintf(propbuf, proplen, "%llu", (u_longlong_t)val);
+ zcp_check(zhp, prop, val, NULL);
break;
default:
switch (zfs_prop_get_type(prop)) {
case PROP_TYPE_NUMBER:
if (get_numeric_property(zhp, prop, src,
- &source, &val) != 0)
+ &source, &val) != 0) {
return (-1);
- if (literal)
+ }
+
+ if (literal) {
(void) snprintf(propbuf, proplen, "%llu",
(u_longlong_t)val);
- else
+ } else {
zfs_nicenum(val, propbuf, proplen);
+ }
+ zcp_check(zhp, prop, val, NULL);
break;
case PROP_TYPE_STRING:
str = getprop_string(zhp, prop, &source);
if (str == NULL)
return (-1);
+
(void) strlcpy(propbuf, str, proplen);
+ zcp_check(zhp, prop, NULL, str);
break;
case PROP_TYPE_INDEX:
@@ -2632,7 +2714,9 @@ zfs_prop_get(zfs_handle_t *zhp, zfs_prop_t prop, char
return (-1);
if (zfs_prop_index_to_string(prop, val, &strval) != 0)
return (-1);
+
(void) strlcpy(propbuf, strval, proplen);
+ zcp_check(zhp, prop, NULL, strval);
break;
default:
Modified: vendor/illumos/dist/lib/libzfs/common/libzfs_impl.h
==============================================================================
--- vendor/illumos/dist/lib/libzfs/common/libzfs_impl.h Wed Sep 13 10:41:47 2017 (r323529)
+++ vendor/illumos/dist/lib/libzfs/common/libzfs_impl.h Wed Sep 13 10:45:49 2017 (r323530)
@@ -79,6 +79,7 @@ struct libzfs_handle {
libzfs_fru_t **libzfs_fru_hash;
libzfs_fru_t *libzfs_fru_list;
char libzfs_chassis_id[256];
+ boolean_t libzfs_prop_debug;
};
struct zfs_handle {
Modified: vendor/illumos/dist/lib/libzfs/common/libzfs_util.c
==============================================================================
--- vendor/illumos/dist/lib/libzfs/common/libzfs_util.c Wed Sep 13 10:41:47 2017 (r323529)
+++ vendor/illumos/dist/lib/libzfs/common/libzfs_util.c Wed Sep 13 10:45:49 2017 (r323530)
@@ -651,6 +651,10 @@ libzfs_init(void)
zpool_feature_init();
libzfs_mnttab_init(hdl);
+ if (getenv("ZFS_PROP_DEBUG") != NULL) {
+ hdl->libzfs_prop_debug = B_TRUE;
+ }
+
return (hdl);
}
Modified: vendor/illumos/dist/lib/libzfs_core/common/libzfs_core.c
==============================================================================
--- vendor/illumos/dist/lib/libzfs_core/common/libzfs_core.c Wed Sep 13 10:41:47 2017 (r323529)
+++ vendor/illumos/dist/lib/libzfs_core/common/libzfs_core.c Wed Sep 13 10:45:49 2017 (r323530)
@@ -20,7 +20,7 @@
*/
/*
- * Copyright (c) 2012, 2014 by Delphix. All rights reserved.
+ * Copyright (c) 2012, 2016 by Delphix. All rights reserved.
* Copyright (c) 2013 Steven Hartland. All rights reserved.
* Copyright (c) 2014 Integros [integros.com]
* Copyright 2017 RackTop Systems.
@@ -152,7 +152,15 @@ lzc_ioctl(zfs_ioc_t ioc, const char *name,
}
while (ioctl(g_fd, ioc, &zc) != 0) {
- if (errno == ENOMEM && resultp != NULL) {
+ /*
+ * If ioctl exited with ENOMEM, we retry the ioctl after
+ * increasing the size of the destination nvlist.
+ *
+ * Channel programs that exit with ENOMEM probably ran over the
+ * lua memory sandbox; they should not be retried.
+ */
+ if (errno == ENOMEM && resultp != NULL &&
+ ioc != ZFS_IOC_CHANNEL_PROGRAM) {
free((void *)(uintptr_t)zc.zc_nvlist_dst);
zc.zc_nvlist_dst_size *= 2;
zc.zc_nvlist_dst = (uint64_t)(uintptr_t)
@@ -864,6 +872,57 @@ lzc_destroy_bookmarks(nvlist_t *bmarks, nvlist_t **err
pool[strcspn(pool, "/#")] = '\0';
error = lzc_ioctl(ZFS_IOC_DESTROY_BOOKMARKS, pool, bmarks, errlist);
+
+ return (error);
+}
+
+/*
+ * Executes a channel program.
+ *
+ * If this function returns 0 the channel program was successfully loaded and
+ * ran without failing. Note that individual commands the channel program ran
+ * may have failed and the channel program is responsible for reporting such
+ * errors through outnvl if they are important.
+ *
+ * This method may also return:
+ *
+ * EINVAL The program contains syntax errors, or an invalid memory or time
+ * limit was given. No part of the channel program was executed.
+ * If caused by syntax errors, 'outnvl' contains information about the
+ * errors.
+ *
+ * ECHRNG The program was executed, but encountered a runtime error, such as
+ * calling a function with incorrect arguments, invoking the error()
+ * function directly, failing an assert() command, etc. Some portion
+ * of the channel program may have executed and committed changes.
+ * Information about the failure can be found in 'outnvl'.
+ *
+ * ENOMEM The program fully executed, but the output buffer was not large
+ * enough to store the returned value. No output is returned through
+ * 'outnvl'.
+ *
+ * ENOSPC The program was terminated because it exceeded its memory usage
+ * limit. Some portion of the channel program may have executed and
+ * committed changes to disk. No output is returned through 'outnvl'.
+ *
+ * ETIME The program was terminated because it exceeded its Lua instruction
+ * limit. Some portion of the channel program may have executed and
+ * committed changes to disk. No output is returned through 'outnvl'.
+ */
+int
+lzc_channel_program(const char *pool, const char *program, uint64_t instrlimit,
+ uint64_t memlimit, nvlist_t *argnvl, nvlist_t **outnvl)
+{
+ int error;
+ nvlist_t *args;
+
+ args = fnvlist_alloc();
+ fnvlist_add_string(args, ZCP_ARG_PROGRAM, program);
+ fnvlist_add_nvlist(args, ZCP_ARG_ARGLIST, argnvl);
+ fnvlist_add_uint64(args, ZCP_ARG_INSTRLIMIT, instrlimit);
+ fnvlist_add_uint64(args, ZCP_ARG_MEMLIMIT, memlimit);
+ error = lzc_ioctl(ZFS_IOC_CHANNEL_PROGRAM, pool, args, outnvl);
+ fnvlist_free(args);
return (error);
}
Modified: vendor/illumos/dist/lib/libzfs_core/common/libzfs_core.h
==============================================================================
--- vendor/illumos/dist/lib/libzfs_core/common/libzfs_core.h Wed Sep 13 10:41:47 2017 (r323529)
+++ vendor/illumos/dist/lib/libzfs_core/common/libzfs_core.h Wed Sep 13 10:45:49 2017 (r323530)
@@ -20,7 +20,7 @@
*/
/*
- * Copyright (c) 2012, 2014 by Delphix. All rights reserved.
+ * Copyright (c) 2012, 2016 by Delphix. All rights reserved.
* Copyright (c) 2014 Integros [integros.com]
* Copyright 2017 RackTop Systems.
*/
@@ -85,6 +85,9 @@ boolean_t lzc_exists(const char *);
int lzc_rollback(const char *, char *, int);
int lzc_rollback_to(const char *, const char *);
+
+int lzc_channel_program(const char *, const char *, uint64_t, uint64_t,
+ nvlist_t *, nvlist_t **);
#ifdef __cplusplus
}
Modified: vendor/illumos/dist/lib/libzpool/common/kernel.c
==============================================================================
--- vendor/illumos/dist/lib/libzpool/common/kernel.c Wed Sep 13 10:41:47 2017 (r323529)
+++ vendor/illumos/dist/lib/libzpool/common/kernel.c Wed Sep 13 10:45:49 2017 (r323530)
@@ -702,6 +702,7 @@ vpanic(const char *fmt, va_list adx)
char buf[512];
(void) vsnprintf(buf, 512, fmt, adx);
assfail(buf, NULL, 0);
+ abort(); /* necessary to make vpanic meet noreturn requirements */
}
void
Modified: vendor/illumos/dist/lib/libzpool/common/sys/zfs_context.h
==============================================================================
--- vendor/illumos/dist/lib/libzpool/common/sys/zfs_context.h Wed Sep 13 10:41:47 2017 (r323529)
+++ vendor/illumos/dist/lib/libzpool/common/sys/zfs_context.h Wed Sep 13 10:45:49 2017 (r323530)
@@ -62,6 +62,7 @@ extern "C" {
#include <time.h>
#include <procfs.h>
#include <pthread.h>
+#include <setjmp.h>
#include <sys/debug.h>
#include <libsysevent.h>
#include <sys/note.h>
@@ -107,8 +108,8 @@ extern void dprintf_setup(int *argc, char **argv);
extern void cmn_err(int, const char *, ...);
extern void vcmn_err(int, const char *, __va_list);
-extern void panic(const char *, ...);
-extern void vpanic(const char *, __va_list);
+extern void panic(const char *, ...) __NORETURN;
+extern void vpanic(const char *, __va_list) __NORETURN;
#define fm_panic panic
@@ -317,6 +318,7 @@ extern void kstat_runq_back_to_waitq(kstat_io_t *);
#define KM_SLEEP UMEM_NOFAIL
#define KM_PUSHPAGE KM_SLEEP
#define KM_NOSLEEP UMEM_DEFAULT
+#define KM_NORMALPRI 0 /* not needed with UMEM_DEFAULT */
#define KMC_NODEBUG UMC_NODEBUG
#define KMC_NOTOUCH 0 /* not needed for userland caches */
#define kmem_alloc(_s, _f) umem_alloc(_s, _f)
Added: vendor/illumos/dist/man/man1m/zfs-program.1m
==============================================================================
--- /dev/null 00:00:00 1970 (empty, because file is newly added)
+++ vendor/illumos/dist/man/man1m/zfs-program.1m Wed Sep 13 10:45:49 2017 (r323530)
@@ -0,0 +1,499 @@
+.\" This file and its contents are supplied under the terms of the
+.\" Common Development and Distribution License ("CDDL"), version 1.0.
+.\" You may only use this file in accordance with the terms of version
+.\" 1.0 of the CDDL.
+.\"
+.\" A full copy of the text of the CDDL should have accompanied this
+.\" source. A copy of the CDDL is also available via the Internet at
+.\" http://www.illumos.org/license/CDDL.
+.\"
+.\"
+.\" Copyright (c) 2016 by Delphix. All Rights Reserved.
+.\"
+.Dd January 21, 2016
+.Dt ZFS-PROGRAM 1M
+.Os
+.Sh NAME
+.Nm zfs program
+.Nd executes ZFS channel programs
+.Sh SYNOPSIS
+.Cm zfs program
+.Op Fl t Ar instruction-limit
+.Op Fl m Ar memory-limit
+.Ar pool
+.Ar script
+.\".Op Ar optional arguments to channel program
+.Sh DESCRIPTION
+The ZFS channel program interface allows ZFS administrative operations to be
+run programmatically as a Lua script.
+The entire script is executed atomically, with no other administrative
+operations taking effect concurrently.
+A library of ZFS calls is made available to channel program scripts.
+Channel programs may only be run with root privileges.
+.Pp
+A modified version of the Lua 5.2 interpreter is used to run channel program
+scripts.
+The Lua 5.2 manual can be found at:
+.Bd -centered -offset indent
+.Lk http://www.lua.org/manual/5.2/
+.Ed
+.Pp
+The channel program given by
+.Ar script
+will be run on
+.Ar pool ,
+and any attempts to access or modify other pools will cause an error.
+.Sh OPTIONS
+.Bl -tag -width "-t"
+.It Fl t Ar instruction-limit
+Execution time limit, in number of Lua instructions to execute.
+If a channel program executes more than the specified number of instructions,
+it will be stopped and an error will be returned.
+The default limit is 10 million instructions, and it can be set to a maximum of
+100 million instructions.
+.It Fl m Ar memory-limit
+Memory limit, in bytes.
+If a channel program attempts to allocate more memory than the given limit, it
+will be stopped and an error returned.
+The default memory limit is 10 MB, and can be set to a maximum of 100 MB.
+.El
+.Pp
+All remaining argument strings will be passed directly to the Lua script as
+described in the
+.Sx LUA INTERFACE
+section below.
+.Sh LUA INTERFACE
+A channel program can be invoked either from the command line, or via a library
+call to
+.Fn lzc_channel_program .
+.Ss Arguments
+Arguments passed to the channel program are converted to a Lua table.
+If invoked from the command line, extra arguments to the Lua script will be
+accessible as an array stored in the argument table with the key 'argv':
+.Bd -literal -offset indent
+args = ...
+argv = args["argv"]
+-- argv == {1="arg1", 2="arg2", ...}
+.Ed
+.Pp
+If invoked from the libZFS interface, an arbitrary argument list can be
+passed to the channel program, which is accessible via the same
+"..." syntax in Lua:
+.Bd -literal -offset indent
+args = ...
+-- args == {"foo"="bar", "baz"={...}, ...}
+.Ed
+.Pp
+Note that because Lua arrays are 1-indexed, arrays passed to Lua from the
+libZFS interface will have their indices incremented by 1.
+That is, the element
+in
+.Va arr[0]
+in a C array passed to a channel program will be stored in
+.Va arr[1]
+when accessed from Lua.
+.Ss Return Values
+Lua return statements take the form:
+.Bd -literal -offset indent
+return ret0, ret1, ret2, ...
+.Ed
+.Pp
+Return statements returning multiple values are permitted internally in a
+channel program script, but attempting to return more than one value from the
+top level of the channel program is not permitted and will throw an error.
+However, tables containing multiple values can still be returned.
+If invoked from the command line, a return statement:
+.Bd -literal -offset indent
+a = {foo="bar", baz=2}
+return a
+.Ed
+.Pp
+Will be output formatted as:
+.Bd -literal -offset indent
+Channel program fully executed with return value:
+ return:
+ baz: 2
+ foo: 'bar'
+.Ed
+.Ss Fatal Errors
+If the channel program encounters a fatal error while running, a non-zero exit
+status will be returned.
+If more information about the error is available, a singleton list will be
+returned detailing the error:
+.Bd -literal -offset indent
+error: "error string, including Lua stack trace"
+.Ed
+.Pp
+If a fatal error is returned, the channel program may have not executed at all,
+may have partially executed, or may have fully executed but failed to pass a
+return value back to userland.
+.Pp
+If the channel program exhausts an instruction or memory limit, a fatal error
+will be generated and the program will be stopped, leaving the program partially
+executed.
+No attempt is made to reverse or undo any operations already performed.
+Note that because both the instruction count and amount of memory used by a
+channel program are deterministic when run against the same inputs and
+filesystem state, as long as a channel program has run successfully once, you
+can guarantee that it will finish successfully against a similar size system.
+.Pp
+If a channel program attempts to return too large a value, the program will
+fully execute but exit with a nonzero status code and no return value.
+.Pp
+.Em Note:
+ZFS API functions do not generate Fatal Errors when correctly invoked, they
+return an error code and the channel program continues executing.
+See the
+.Sx ZFS API
+section below for function-specific details on error return codes.
+.Ss Lua to C Value Conversion
+When invoking a channel program via the libZFS interface, it is necessary to
+translate arguments and return values from Lua values to their C equivalents,
+and vice-versa.
+.Pp
+There is a correspondence between nvlist values in C and Lua tables.
+A Lua table which is returned from the channel program will be recursively
+converted to an nvlist, with table values converted to their natural
+equivalents:
+.Bd -literal -offset indent
+string -> string
+number -> int64
+boolean -> boolean_value
+nil -> boolean (no value)
+table -> nvlist
+.Ed
+.Pp
+Likewise, table keys are replaced by string equivalents as follows:
+.Bd -literal -offset indent
+string -> no change
+number -> signed decimal string ("%lld")
+boolean -> "true" | "false"
+.Ed
+.Pp
+Any collision of table key strings (for example, the string "true" and a
+true boolean value) will cause a fatal error.
+.Pp
+Lua numbers are represented internally as signed 64-bit integers.
+.Sh LUA STANDARD LIBRARY
+The following Lua built-in base library functions are available:
+.Bd -literal -offset indent
+assert rawlen
+collectgarbage rawget
+error rawset
+getmetatable select
+ipairs setmetatable
+next tonumber
+pairs tostring
+rawequal type
+.Ed
+.Pp
+All functions in the
+.Em coroutine ,
+.Em string ,
+and
+.Em table
+built-in submodules are also available.
+A complete list and documentation of these modules is available in the Lua
+manual.
+.Pp
+The following functions base library functions have been disabled and are
+not available for use in channel programs:
+.Bd -literal -offset indent
+dofile
+loadfile
+load
+pcall
+print
+xpcall
+.Ed
+.Sh ZFS API
+.Ss Function Arguments
+Each API function takes a fixed set of required positional arguments and
+optional keyword arguments.
+For example, the destroy function takes a single positional string argument
+(the name of the dataset to destroy) and an optional "defer" keyword boolean
+argument.
+When using parentheses to specify the arguments to a Lua function, only
+positional arguments can be used:
+.Bd -literal -offset indent
+zfs.sync.destroy("rpool at snap")
+.Ed
+.Pp
+To use keyword arguments, functions must be called with a single argument that
+is a Lua table containing entries mapping integers to positional arguments and
+strings to keyword arguments:
+.Bd -literal -offset indent
+zfs.sync.destroy({1="rpool at snap", defer=true})
+.Ed
+.Pp
+The Lua language allows curly braces to be used in place of parenthesis as
+syntactic sugar for this calling convention:
+.Bd -literal -offset indent
+zfs.sync.snapshot{"rpool at snap", defer=true}
+.Ed
+.Ss Function Return Values
+If an API function succeeds, it returns 0.
+If it fails, it returns an error code and the channel program continues
+executing.
+API functions do not generate Fatal Errors except in the case of an
+unrecoverable internal file system error.
+.Pp
+In addition to returning an error code, some functions also return extra
+details describing what caused the error.
+This extra description is given as a second return value, and will always be a
+Lua table, or Nil if no error details were returned.
+Different keys will exist in the error details table depending on the function
+and error case.
+Any such function may be called expecting a single return value:
+.Bd -literal -offset indent
+errno = zfs.sync.promote(dataset)
+.Ed
+.Pp
+Or, the error details can be retrieved:
+.Bd -literal -offset indent
+errno, details = zfs.sync.promote(dataset)
+if (errno == EEXIST) then
+ assert(details ~= Nil)
+ list_of_conflicting_snapshots = details
+end
+.Ed
+.Pp
+The following global aliases for API function error return codes are defined
+for use in channel programs:
+.Bd -literal -offset indent
+EPERM ECHILD ENODEV ENOSPC
+ENOENT EAGAIN ENOTDIR ESPIPE
+ESRCH ENOMEM EISDIR EROFS
+EINTR EACCES EINVAL EMLINK
+EIO EFAULT ENFILE EPIPE
+ENXIO ENOTBLK EMFILE EDOM
+E2BIG EBUSY ENOTTY ERANGE
+ENOEXEC EEXIST ETXTBSY EDQUOT
+EBADF EXDEV EFBIG
+.Ed
+.Ss API Functions
+For detailed descriptions of the exact behavior of any zfs administrative
+operations, see the main
+.Xr zfs 1
+manual page.
+.Bl -tag -width "xx"
+.It Em zfs.debug(msg)
+Record a debug message in the zfs_dbgmsg log.
+A log of these messages can be printed via mdb's "::zfs_dbgmsg" command, or
+can be monitored live by running:
+.Bd -literal -offset indent
+ dtrace -n 'zfs-dbgmsg{trace(stringof(arg0))}'
+.Ed
+.Pp
+msg (string)
+.Bd -ragged -compact -offset "xxxx"
+Debug message to be printed.
+.Ed
+.It Em zfs.get_prop(dataset, property)
+Returns two values.
+First, a string, number or table containing the property value for the given
+dataset.
+Second, a string containing the source of the property (i.e. the name of the
+dataset in which it was set or nil if it is readonly).
+Throws a Lua error if the dataset is invalid or the property doesn't exist.
+Note that Lua only supports int64 number types whereas ZFS number properties
+are uint64.
+This means very large values (like guid) may wrap around and appear negative.
+.Pp
+dataset (string)
+.Bd -ragged -compact -offset "xxxx"
+Filesystem or snapshot path to retrieve properties from.
+.Ed
+.Pp
+property (string)
+.Bd -ragged -compact -offset "xxxx"
+Name of property to retrieve.
+All filesystem, snapshot and volume properties are supported except
+for 'mounted' and 'iscsioptions.'
+Also supports the 'written at snap' and 'written#bookmark' properties and
+the '<user|group><quota|used>@id' properties, though the id must be in numeric
+form.
+.Ed
+.El
+.Bl -tag -width "xx"
+.It Sy zfs.sync submodule
+The sync submodule contains functions that modify the on-disk state.
+They are executed in "syncing context".
+.Pp
+The available sync submodule functions are as follows:
+.Bl -tag -width "xx"
+.It Em zfs.sync.destroy(dataset, [defer=true|false])
+Destroy the given dataset.
+Returns 0 on successful destroy, or a nonzero error code if the dataset could
+not be destroyed (for example, if the dataset has any active children or
+clones).
+.Pp
+dataset (string)
+.Bd -ragged -compact -offset "xxxx"
+Filesystem or snapshot to be destroyed.
+.Ed
+.Pp
+[optional] defer (boolean)
+.Bd -ragged -compact -offset "xxxx"
+Valid only for destroying snapshots.
+If set to true, and the snapshot has holds or clones, allows the snapshot to be
+marked for deferred deletion rather than failing.
+.Ed
+.It Em zfs.sync.promote(dataset)
+Promote the given clone to a filesystem.
*** DIFF OUTPUT TRUNCATED AT 1000 LINES ***
More information about the svn-src-all
mailing list