Re: Switching release media dist sets to .tzst (tar + zstd)?

From: Zhenlei Huang <zlei_at_FreeBSD.org>
Date: Sat, 14 Dec 2024 01:58:16 UTC

> On Dec 14, 2024, at 5:15 AM, Ed Maste <emaste@freebsd.org> wrote:
> 
> I have been reviewing parts of the release artifact build process,
> including ISO and memstick images, and came across the distribution
> sets (e.g., base.txz, src.txz) used by the installer to populate new
> file systems. I’d like to discuss switching these to .tzst (tar +
> zstd) compression.
> 
> While I haven’t yet conducted detailed benchmarks comparing zstd and
> xz specifically for this use case, here are some initial
> considerations:
> 
> Pros of zstd:
> - Faster compression and decompression speeds.
> - Aligns with the compression method used for FreeBSD packages.
> 
> Cons of zstd:
> - Somewhat larger compressed file sizes.
> - Requires updates to tools that interact with distribution sets.
> - May have limited availability on some other operating systems (?).
> 
> I have a review open to demonstrate the extent of the change in the
> build system & installer: https://reviews.freebsd.org/D48042
> 
> It might be that this is not worth pursuing, as dist sets will most
> likely go away with the migration to pkgbase, but I would like to
> discuss and make an explicit decision. We can separately consider
> compression on the release media images themselves.
> 
> Feedback Requested:
> 
> Is there support for this idea? Are there objections to pursuing this?
> Are there other factors I should consider, especially compatibility concerns?
> 


I have slow internet, I'd always prefer xz than zstd, as the decompress speed does
not matter much.

For the compression, I'm not member of RE, but I guess that ( the slow speed of 
compression ) is affordable, as the releasing is not frequent.

One good example usage of zstd is compressed kernel ( it seems we do not 
support that yet ). OCI images may also have benefit with that as those images
typically not large, xz saves a little more spaces then zstd but the decompress
speed is much slower. It is good to have fast startup speed of containers.

Best regards,
Zhenlei