Fwd: ZFS Encryption Implementation for Review

[Andriy Gapon]

Just in case people overlooked this information in another thread here.

-------- Forwarded Message --------
Subject: [developer] ZFS Encryption Implementation for Review
Date: Tue, 17 May 2016 17:17:53 -0400
From: Thomas Caputi <tcaputi at datto.com>
To: developer at open-zfs.org

I have created an implementation for native encryption in ZFS. This
implementation is currently available as a PR against ZoL
(https://github.com/zfsonlinux/zfs/pull/4329). I would appreciate it
if this PR could receive a review for consideration. For convenience,
I have pasted the PR's description below.

Thanks,
Tom Caputi

Native encryption in zfsonlinux (See issue #494)

The change incorporates 3 major pieces:

The first is a port of the Illumos Crypto Framework to a Linux kernel
module (found in module/icp). This is needed to do the actual
encryption work. We cannot use the Linux kernel's built in crypto api
because it is only exported to GPL-licensed modules. Having the ICP
also means the crypto code can run on any of the other kernels under
OpenZFS. I ended up porting over most of the internals of the
framework, which means that porting over other API calls (if we need
them) should be fairly easy. Specifically, I have ported over the API
functions related to encryption, digests, macs, and crypto templates.
The ICP is able to use assembly-accelerated encryption on amd64
machines and AES-NI instructions on Intel chips that support it. There
are place-holder directories for similar assembly optimizations for
other architectures (although they have not been written).

The second feature is a keystore that manages wrapping and encryption
keys for encrypted datasets. It has feature parity with Solaris, but
should be more predictable and consistent. It is fully integrated with
the existing zfs create functions and zfs clone functions. It also
exposes a new set of commands via zfs key for managing the keystore.
For more info on the inconsistencies in Solaris see my comment
(https://github.com/zfsonlinux/zfs/issues/494#issuecomment-178853634)
on the issue page. The keystore operates on a few rules:

All wrapping keys are 32 bytes (256 bits), even for 128 and 192 bit
encryption types.

Encryption must be specified at dataset creation time.

Specifying a keysource while creating a dataset causes the dataset to
become the root of an encryption tree.

All members of an encryption tree share the same wrapping key.

Each dataset can have up to 1 keychain (if it is encrypted) that is
not shared with anybody.


The last feature is the actual data and metadata encryption. All data
in an encrypted dataset is stored encrypted on-disk. User-provided
metadata is also encrypted, but metadata structures have been left
plain so that scrubbing and resilvering still works without the keys
loaded. Most of the design comes from this article
(https://blogs.oracle.com/darren/entry/zfs_encryption_what_is_on).
There are a few important distinctions, however. For instance, I store
the encryption IV in the padding of blkptr_t instead of in its third
DVA. I also have L2ARC encryption implemented, which Oracle did not
have at the time.


Implementation details that should be looked at

I created a new DMU_OT_* for keychain objects instead of using the
DMU_OTN() macro. I did this mostly for the ability to to register a
name which helped with debugging. The Keychain objects also seem like
a core enough structure to warrant a new dedicated object type.

The crypto framework has some code bloat to it, particularly in the
form of function stubs in header files that are never actually
implemented. I figured it would be best to leave these in, in case
more functions needed to be ported over.

The in-memory keystore is not the most efficient structure, since it
zeros and frees encryption keys whenever they are not in use. This is
intended as a security measure so that unwrapped keys do not exist in
memory longer than they are needed.

Encrypting data going to disk requires creating a keychain_record_t
during dsl_dataset_tryown(). I added a flag to this function for code
that wishes to own the dataset, but that does not require encrypted
data, such as the scrub functions. I did my best to confirm that all
owners set this flag correctly, but someone should confirm them, just
to be sure.

zfs send and zfs recv do not currently do anything special with
regards to encryption. The format of the send file has not changed and
zfs send requires the keys to be loaded in order to work. At some
point there should probably be a way to do encrypted sends.

I altered the prototype of lzc_create() and lzc_clone(). I understand
that the purpose of libzfs_core is to have a stable api interacting
with the ZFS ioctls. However, these functions need to accept wrapping
keys separately from the rest of their parameters because they need to
use the (new) hidden_args framework to support hiding arguments from
the logs. Without this, the wrapping keys would get printed to the
zpool history.

There is an extra local label that I needed to add to the top of of
the "global" function rijndael_key_setup_enc_intel() in
module/icp/aes_intel.S. For some reason, I had to use the local label
or the module would fail to link. If any assembly experts can tell me
why this is required or a better way to fix it, I would appreciate it.

As part of the L2ARC changes, I added a 8 byte MAC field to
l2arc_buf_hdr_t. I understand that there are a lot of reasons to keep
this struct small since many of them may be allocated at once, but I
do not seem to have another reasonable option here.

The icp is a kernel module that has a directory structure (unlike the
other modules in zfs). There are a few reasons for this. First, the
ICP has assembly code for different CPU architectures and I wanted to
match the structure of libspl. The ICP also has headers that did not
really need to belong in the global zfs headers and so it made sense
to make an includes directory for them locally. The directory
structure also approximately mimics the the structure of the Illumos
Crypto Framework, which will be important for maintainability. As a
result, I had to adjust the build systems to avoid flattening module
directories. This shouldn't matter much, since the other modules were
already flat.


-------------------------------------------
openzfs-developer
Archives: https://www.listbox.com/member/archive/274414/=now
RSS Feed: https://www.listbox.com/member/archive/rss/274414/28133750-22ed9730
Modify Your Subscription:
https://www.listbox.com/member/?member_id=28133750&id_secret=28133750-6c0d6209
Powered by Listbox: http://www.listbox.com