Re: Plan for "distribution set" deprecation

On Mon, Sep 22, 2025 at 11:54:30AM -0700, Colin Percival wrote:
> Hi everyone,
> 
> With pkgbase landing in 15.0, I decided that it's time to announce the
> timeline for deprecation of the legacy "distribution sets".  Here's the
> plan:
> 
> 15.0-RELEASE:
> * PKGBASE becomes the default in release/Makefile and a new NOPKGBASE
> option is added.
> * The installer supports both install options, with dist sets marked
> as "legacy" in the pick-how-to-install menu.
> * The disc1 images only contain a pkgbase repo; people who want to install
> from disc1 without pkgbase will need an internet connection.
> * The dvd images contain both dist sets and a pkgbase repo suitable for
> offline installation with either mechanism.
> * Traditional FreeBSD Update will work for the entire 15.x branch, including
> security updates, since this is relatively easy to do as long as we have dist
> sets.
> * Work is underway to make freebsd-update behave as a wrapper around pkgbase
> on pkgbase-enabled systems (right now it just refuses to do anything).
> 
> 16-CURRENT around 2026Q4:
> * NODISTSETS becomes the default in release/Makefile and a new DISTSETS
> option is added to enable them.
> * support for dist sets in the installer is disabled by default (build
> option to re-enable).
> 
> 16.0-RELEASE and later:
> * Legacy distribution sets are not included on any release media.
> * Install images only support pkgbase.
> * Legacy FreeBSD Update does not exist on 16.x; users will need to
> pkgbasify their 15.x systems before upgrading to 16.x.
> * Code for distribution sets remains in the tree so that it can be used
> by downstream projects which aren't on pkgbase yet.
> 
> 17-CURRENT around 2028Q4:
> * All the dist set code gets removed from main.
> 
> Note that this is all about releases and binary updates; the traditional
> process for updating from source (installworld and friends) is not expected
> to change in the near future.  It's possible that *eventually* that will
> move to a "stage the world, package it, and then install the packages"
> approach (just like ports moved from installing directly to "staging" over
> a decade ago) but if that change happens there will be ample notice.

Hey Colin,

Thank you for your (and others!) work on this. I'm hoping that this
bug can be resolved before PKGBASE is enabled by default for release
generation: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=287877

For the life of me, I can't figure out why I'm hitting that bug
downstream. I've made the changes I need to make to match on
HardenedBSD-related packages (and can provide the diff if needed.) But
even with those changes (mainly just `sed s/FreeBSD/HardenedBSD/g`)
I'm still hitting that lua assertion.

I haven't tried on regular FreeBSD, only HardenedBSD. But I suspect
that since another FreeBSD user is hitting that assertion, it's likely
not related to HardenedBSD.

I have confirmed that the pkgbase repo that gets built does indeed
include the right kernel package. Hence why I'm puzzled why that lua
assertion is being tripped.

This is the one thing preventing HardenedBSD from being a guinea pig
in this pkgbase effort.

Update after writing the bulk of the email: I just retried making
release media (previous attempt was just a day or two before that bug
report having been submitted). I hit the same assertion, but the
release/scripts/pkgbase-stage.lua script has changed. New line number:
38. Below is the HardenedBSD version of the script (again, mainly just
a branding change). The line numbers are included at the very left.

==== BEGIN lua script OUTPUT ====
  22 local function select_packages(pkg, media, all_libcompats)
  23         local components = {}
  24         local rquery = capture(pkg .. "rquery -U -r HardenedBSD-base %n")
  25         for package in rquery:gmatch("[^\n]+") do
  26                 local set = package:match("^HardenedBSD%-set%-(.*)$")
  27                 if set then
  28                         components[set] = package
  29                 -- Kernels other than FreeBSD-kernel-generic are ignored
  30                 -- Note that on powerpc64 and powerpc64le the names are
  31                 -- slightly different.
  32                 elseif package:match("^HardenedBSD%-kernel%-hardenedbsd.*-dbg") then
  33                         components["kernel-dbg"] = package
  34                 elseif package:match("^HardenedBSD-kernel%-hardened.*") then
  35                         components["kernel"] = package
  36                 end
  37         end
  38         assert(components["kernel"])
  39         assert(components["base"])
  40 
  41         local selected = {}
  42         if media == "disc" then
==== END lua script OUTPUT ====

That assertion, the assert(components["kernel"]) is what fails, even
though the package is actually there.

Thanks,

-- 
Shawn Webb
Cofounder / Security Engineer
HardenedBSD

Signal Username:  shawn_webb.74
Tor-ified Signal: +1 303-901-1600 / shawn_webb_opsec.50
https://git.hardenedbsd.org/hardenedbsd/pubkeys/-/raw/master/Shawn_Webb/03A4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc