Re: [HEADS UP] stable/15: new base packages for PAM, zstd

Kurt Jaeger <pi@freebsd.org> writes:
> From what I've learned, initial installation of 'base' from packages
> is roughly the same as before, but *updates* are easier to handle.

* Installing on bare metal (or a VM) is roughly the same, but creating a
  jail is much easier.

* Updates are easier to handle, both on the provider side and on the
  consumer side.  The downside is that pkgbase consumes significantly
  more bandwidth than freebsd-update, especially for security updates.

* The system can be stripped down to the bare essentials without losing
  the ability to receive updates.  The smallest usable set you can
  maintain with freebsd-update is base = over 700 MB or almost 900 MB if
  you add a kernel.  The smallest usable set you can maintain with
  pkgbase is clibs + runtime (+ rc) = less than 15 MB or 175 MB with a
  kernel.

> If you look at the freebsd-update script, the run time to find which
> files need to be updated etc was always resource-intensive.

The problem with the freebsd-update client is mostly that it is a shell
script.  There is an unofficial Rust implementation which does the exact
same job much, much faster, though I can't vouch for it.

> I have no insight in the backend of freebsd-update, maybe Colin
> can describe the pros and cons from that side ?

We first do a full release build of each patched branch.  Then the
freebsd-update build scripts download the build artifacts and compare
them to the previous version to inventory added, deleted, and modified
files and produce binary diffs for each and indices for the whole thing.
This time-consuming process is repeated for the cartesian join of every
version we want to be able to update _from_ (all supported releases plus
a handful of recently-out-of-support releases) and every version we want
to be able to update _to_ (all supported releases).  The diffs are then
signed and pushed to the mirrors, which also takes several hours.  Only
when all mirrors have synced the full set can we push an additional tiny
diff that makes the updates visible to users (and only takes minutes to
propagate).

I realize it sounds quite mad but keep in mind that this was first
deployed 20 years ago and the primary objective was to minimize the
bandwidth consumed by updates, which this system does _very well_.
FreeBSD is much bigger today than it was 20 years ago, but only by a
factor of around 10, while bandwidth has increased by a factor of about
1,000 over the same period of time.

DES
--
Dag-Erling Smørgrav - des@FreeBSD.org