ZSTD Project Weekly Status Update

[Allan Jude]

This is the eleventh weekly status report on my FreeBSD Foundation
sponsored project to complete the integration of ZSTD compression into
OpenZFS.

As I continue to work on the future-proofing issue, I have also been
lending a hand to the integration of OpenZFS into FreeBSD, and doing a
bunch of reviewing and testing there.

I have also been doing some benchmarking of the ZSTD feature.

so far I have tried 4 different approaches with varying results.

The test rig:
A single socket Intel(R) Xeon(R) CPU E5-2630 v4 @ 2.20GHz (10 cores, 20
threads)
32 GB ram
ZFS ARC max = 4GB
4x Samsung 860 EVO SSDs


1) using fio. This gives slightly more useful output, both bandwidth and
IOPS but also has more detail about changes over time as well as latency
etc.

The downside here is that it doesn't really benchmark compression. By
default fio uses fully random data that does not compress at all. This
is a somewhat useful metric, and the differing results seen when varying
blocksize is interesting.

fio has an option, --buffer_compress_percentage=, to select how
compressible you want the generated data to be. However, this just
switches between random data, and a repeating pattern (by default null
bytes). So different levels of zstd compression all give the same
compression ratio (the level you ask fio to generate). This doesn't
really provide the real-work use case of having a tradeoff where
spending more time on compression results in a greater space savings.

2) I also used 'zfs recv' to create more repeatable writes. I generated
a large dataset, 8 copies of the FreeBSD 12.1 source code, that rounds
out to around 48 GB of uncompressed data, snapshoted it, and created a
zfs send stream, stored on a tmpfs. Then I measured the time taken to
zfs recv that stream, at different compression levels. I later also
redid these experiments at different record sizes.

The reason I chose to use 8 copies of the data was to make the runs long
enough at the lower compression levels to get more consistent readings.

The issue with this was also a limitation of my test setup, 4x striped
SSDs, that tends to top out around 1.3 GB/sec of writes. So the
difference between compression=off, lz4, and zstd-1 was minimal.

3) I then the zfs recv based testing, but with only 1 copy of the source
code (1.3 GB) but with the pool backed by a file on a tmpfs. Removing
the SSDs from the equation. The raw write speed to the tmpfs was around
3.2GB/sec.

4) I also redid the fio based testing with a pool backed by a file on tmpfs.


I am not really satisfied with the quality of the results so far.

Does Linux have something equivalent to FreeBSD's mdconfig, where I can
create an arbitrarily number of arbitrarily sized memory-backed devices,
that I could use to back the pool? A file-based vdev on a tmpfs just
doesn't seem to provide the same type of results as I was expecting.

Any other suggestions would be welcome.



In the end the results will all be relative, which is mostly what we are
looking to capture. How much faster/slow is zstd at different levels
compared to lz4 and gzip, and how much more compression do you get in
exchange for that trade-off.

Hopefully next week there will be some pretty graphs.

Thanks again to the FreeBSD Foundation for sponsoring this work.


On 2020-08-25 22:22, Allan Jude wrote:
> This is the tenth weekly status report on my FreeBSD Foundation
> sponsored project to complete the integration of ZSTD compression into
> OpenZFS.
> 
> Late last week the main pull request was merged, and ZSTD support is now
> part of OpenZFS's trunk branch.
> 
> Last night, OpenZFS with ZSTD was imported into FreeBSD's -current branch.
> 
> I am continuing to work on a number of things related to ZSTD, including
> future-proofing support (so upgrading ZSTD won't cause problems with
> features like nopwrite), and improving the integration of ZSTD into
> FreeBSD, including enabling support for booting from ZSTD compressed
> datasets, and improving the performance of ZSTD on FreeBSD.
> 
> I'll also be adding some additional tests to make sure we detect any
> issues when we do look at updating ZSTD. Additionally, I am working on a
> bunch of documentation around using ZSTD in ZFS.
> 
> For my benchmarking of ZSTD, I have been using a zfs recv of a stream in
> a file on a tmpfs, and recording how long it takes to receive and sync
> the data. The test data is a copy of the FreeBSD 12.1 source code, since
> that is easily reproducible.
> 
> Does anyone have experience or a better suggestion on how to get the
> most consistent and repeatable results when benchmarking like this?
> 
> 
> On 2020-08-18 18:51, Allan Jude wrote:
>> This is the ninth weekly status report on my FreeBSD Foundation
>> sponsored project to complete the integration of ZSTD compression into
>> OpenZFS.
>>
>> https://github.com/openzfs/zfs/pull/10693 - The L2ARC fixes (for when
>> compressed ARC is disabled) have been merged.
>>
>> https://github.com/openzfs/zfs/pull/10278/ - A number of other cleanups
>> and fixes for the ZSTD have been integrated and squashed, and it looks
>> like the completed ZSTD feature will be merged very soon.
>>
>> This included a bunch of fixes for makefiles and runfiles to hook the
>> tests I added up to the ZFS test suite so they are run properly.
>>
>> It looks like this will mean that the ZSTD feature will be included in
>> OpenZFS 2.0. Thanks for everyone who has tested, reviewed, or
>> contributed to this effort, especially those who kept it alive while I
>> was working on other things.
>>
>> Post-merge, the remaining work is to develop future-proofing around ZSTD
>> so that we will be able to more seamlessly upgrade to newer releases of
>> ZSTD. Recompression of the same input resulting in the same on-disk
>> checksum is the main concern, as without this upgrading the compression
>> algorithm will break features like nop-write.
>>
>> This project is sponsored by the FreeBSD Foundation.
>>
> 
> 


-- 
Allan Jude

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 834 bytes
Desc: OpenPGP digital signature
URL: <http://lists.freebsd.org/pipermail/freebsd-fs/attachments/20200831/9d1d8110/attachment.sig>