Relocating the zfs zpool.cache file

[grarpamp]

Given a system with all of the base file systems mounted read-only
for change control and security auditing, how does one relocate
zpool.cache? ZFS SPA keeps trying to create it and tries to update
it on occaision.

What does zpool.cache do for us if everything is kept tasteable on
disk already? Seems that without this file, the zpools cannot be
imported without the use of the -f in zpool import -f. Not sure why
that is.

The system uses geom_eli with a keyboard entered token component,
so automounting the zfs datasets through rc at boot is not possible.

# read-only ufs
/, /usr, /usr/local
/z    [mountpoints for zfs datasets, tiny 32m partition]
# read-write ufs
/var  [varmfs_flags=-SM]
/tmp  [tmpmfs_flags=-SM]
# read-write zfs
/z/*  [zfs datasets]

Was thinking about making /z read-write and putting zpool.cache there.
Symlinking /boot/zfs/zpool.cache -> /z/zpool.cache does not work.
I guess /boot/zfs could be made into another tiny filesystem.


### references to the zpool cache in the source
Presumably the loader.conf bits load the zpool config into the zfs
kernel module so that things can mount automatically on boot?

The zfsimpl.h and zfs.h defines are unfortunately not boot time
tuneables. Is it possible to make the zfs.h ZPOOL_CACHE refer to
the zfsimpl.h definition, or the other way around?

ZPOOL_CACHE_TMP doesn't appear to be used or necessary as the
function  'spa_config.c: spa_config_write'  seems to be the intended
use.

# from zfsimpl.h - except that 'zpool get cachefile <pool>' returns
#  invalid property
Pools can also have the 'cachefile' property set that allows them
to be stored in an alternate location until the control of external
software.


# find -sEx src -type f | xargs egrep -i 'zpool(\.|_)cache|spa_config_path'
src/cddl/contrib/opensolaris/cmd/zdb/zdb.c:
spa_config_path = optarg;
src/cddl/contrib/opensolaris/cmd/zdb/zdb.c:
dump_cachefile(spa_config_path);
src/cddl/contrib/opensolaris/cmd/ztest/ztest.c:     "/usr/sbin%.*s/zdb
-bc%s%s -U /tmp/zpool.cache -O %s %s",
src/cddl/contrib/opensolaris/cmd/ztest/ztest.c: /* Override location
of zpool.cache */
src/cddl/contrib/opensolaris/cmd/ztest/ztest.c: spa_config_path =
"/tmp/zpool.cache";
src/cddl/contrib/opensolaris/cmd/ztest/ztest.c:  * Blow away any
existing copy of zpool.cache
src/cddl/contrib/opensolaris/cmd/ztest/ztest.c:         (void)
remove("/tmp/zpool.cache");
src/cddl/contrib/opensolaris/lib/libzfs/common/libzfs.h:
ZPOOL_STATUS_CORRUPT_CACHE,     /* corrupt /kernel/drv/zpool.cache */
src/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_config.c: * The
pool configuration repository is stored in /etc/zfs/zpool.cache as a
src/sys/boot/forth/loader.conf:zpool_cache_load="YES"
src/sys/boot/forth/loader.conf:zpool_cache_type="/boot/zfs/zpool.cache"
src/sys/boot/forth/loader.conf:zpool_cache_name="/boot/zfs/zpool.cache"
src/sys/cddl/boot/zfs/zfsimpl.h:#define ZPOOL_CACHE_DIR         "/boot/zfs"
src/sys/cddl/boot/zfs/zfsimpl.h:#define ZPOOL_CACHE_FILE        "zpool.cache"
src/sys/cddl/boot/zfs/zfsimpl.h:#define ZPOOL_CACHE_TMP         ".zpool.cache"
src/sys/cddl/boot/zfs/zfsimpl.h:#define ZPOOL_CACHE
ZPOOL_CACHE_DIR "/" ZPOOL_CACHE_FILE
src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c:
 } else if (strcmp(dp->scd_path, spa_config_path) != 0) {
src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c:
                 dp->scd_path = spa_strdup(spa_config_path);
src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa_config.c: *
default, all pools are stored in /etc/zfs/zpool.cache and loaded on
boot
src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa_config.c: *
from /etc/zfs/zpool.cache and populate the SPA namespace.  This
namespace is
src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa_config.c:const
char *spa_config_path = ZPOOL_CACHE;
src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa_config.c:
 (void) snprintf(pathname, MAXPATHLEN, "%s", spa_config_path);
src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa_config.c: *
Sigh.  Inside a local zone, we don't have access to
/etc/zfs/zpool.cache,
src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa_misc.c:
dp->scd_path = spa_strdup(spa_config_path);
src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/spa_impl.h:extern
const char *spa_config_path;
src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c: *
/etc/zfs/zpool.cache was readonly at the time.  Otherwise, the vdev
state
src/sys/cddl/contrib/opensolaris/uts/common/sys/fs/zfs.h:#define
 ZPOOL_CACHE             "/boot/zfs/zpool.cache"