HEADSUP: FLAVORS (initial version) and subpackages proposals

Thu Dec 22 20:04:55 UTC 2016

On Tue, Dec 20, 2016 at 07:11:11PM +1100, Dewayne Geraghty wrote:
> Thanks Bapt et al,
> 
> I use FreeBSD and the ports system extensively, we build everything from
> source and largely customise approx 25% of the 900 packages we rely upon.
> I'm more than a little concerned to have changes performed against the
> ports infrastructure.  As our primary sources of (whats coming) "Change"
> information are the: Quarterly reports and the OS Release Notes;
> after-the-fact sources are a daily review of
> https://lists.freebsd.org/pipermail/svn-src-stable-11/2016-December/thread.html
> for OS impact; and the excellent Freshports.
> 
> So a few questions:
> 
> Could you be able to enlighten us (the readers) so we can better understand
> what will be changed; or share your vision of the benefits and operational
> impact for operational people that build: from source; and those that only
> use binary?

Sure so there are 2 different things that are requested for a long time by lots
of users:
1/ flavors

This is the avility to say this port should be built with a list of
variations by default. There are 2 kind of flavors:
a/ the one that are not conflicting: for example being able to have all python
   ports available for both python 2 and python 3 by default and at the same
   time. Right now it is not directly possible without a hack
b/ the one that makes variations of a given package right now done a hackish way
   via the (for example) *-nox11, *-static or *-lite.

The goal of the change I do propose aims at making both kind of flavors easily
maintainable.

The difficulty is bringing in that feature without breaking anything for end
users.

The clean way would be to to just have a new variable in a given port that
describes the possible variations. But that would break all existing external
tools that deals with the ports tree. Because they all rely on the fact that
there is a mapping between a package name and an origin (not that pkg does not
rely on that.

It means that for poudriere and synth we can easily adapt them, both are very
actively maintained and their design should allow "easily" to integrate that
But portmaster and portupgrade would be deeply broken as not actively
maintained

So I decided to go another way: add a third level to the ports tree. So far we
have category/port and I do propose to add a third level: category/port/flavor
which will keep the paradigm most tools are expected: 1 packagename == 1 origin

Maybe some tools would have to be updated a bit, but that would be minor patches

With my current patch the only problem I see that the category/port level is
unused if there are flavors while I could certainly make it "the default flavor"

the drawback of this approach is it will add a lots of new small files and
directory.

The most important part of the flavors is probably the ability to provide
natively support for python2 and python3 at the same time

A good side effect of adding a third level is we could now imagine regrouping
some ports (the openbsd ports tree does that already) like aspell where we could
have:
textproc/aspell and textproc/aspell/fr textproc/aspell/en etc which would make
things a bit cleaner

2/ sub packages

sub packages is the ability from one build to create multiple packages.
The goal is to avoid the giant gcc package as a runtime dependency for example
where we could provide a gcc-libs package for the runtime libary

It would also allow to save a lot of building time for things like php or qt We
could end up with on single port (so one single extraction and one single run of
the configure script) while keeping the flexbility of the current split which is
existing right now in the ports tree with zillion of complex slave ports.

The big issue while designing that solution is it cannot be made transparent for
the users as it will for sure break the paradigm: 1 pkgname == 1 origin

> 
> Is there a transition plan or schedule for the bulk of these changes to
> occur?

For flavours it should be transparent if not that would be a bug except if
everyone argue I should break the paradigm 1 pkgname == 1 origin and go for the
clean implementation
> 
> Will the flavors/subpackages be developed separately from the existing
> ports suite?  (I'm hoping that the parent ports will be unaffected, and so
> our existing build procedures continue to build correctly)

I don't see how it can be developped separately, can you elaborate more?
> 
> How will we (the users/admins) track or be informed of changes or better,
> planned/soon changes?  (will changes to ports, particularly parent ports,
> be co-ordinated through UPDATING or perhaps a new FLAVOURS file if the
> parent is say a stub and the real decisions are relocated to slaves?)

Yes of course UPDATING and MOVED when needed
for example shells/bash-static|shells/bash/static|....

> 
> Will there be any guidance regarding how flavours/packages should be
> created or the criteria for creating sub-packages (secure/insecure; all
> options on/off; most useable options on; most liked by the maintainer; most
> likely to be used for a datacentre; most likely to be used for desktops;
> ...)?   Will "The Porter's Handbook" be updated for things like criteria;
> naming conventions etc?

There won't me less or more guidance than nowaday. I'm not in portmgr anymore so
maybe portmgr will decide differently :)
> 
> For folks (like me) that build entirely from source and customise options
> to build the applications, how will flavours/subpackages be of benefit?
> Will the ability to customise ports, as they exist today, remain?  Will I
> even notice a change?

flavours and subpackages will be a benefit because it will reduce the
requirement to have OPTIONS (for some case) meaning you will have less work to
get the same level of flexibility you having now (probably even more)

For non source builder they will benefit more flexibility than they have now.
> 
> I'd like to plan ahead to make this transition seemless and continue to use
> FreeBSD and the excellent ports system as we do now.

As far as I remember I have always worked hard on making the majors changes I
brought in the ports tree as seemless as possible for users (the replacement of
the option framework for example, the complete rewrite of how LIB_DEPENDS is
handled). I will do the same for those features even if that sounds very hard to
achieve for subpackages for now.
> 
> I started with FreeBSD 2.2.8.  There were packages available from the
> FreeBSD website.  It was a terrific aid.  We also enjoyed the different
> flavours of jail that were provided by ezjail.  However over time, both
> evolved as did our expertise to customise our ports (~200 custom ports) and
> Jamie Gratton evolved the jail system to eliminate our need of the
> excellent ezjail tool.  So I can see merit in, what very little I'm
> guessing of, the next evolution of ports.
> 
> Aside: we already build different package configurations from existing
> ports' source.  (eg different bind910 with/without kerberos; different
> samba44's; simultaineous building of dhcp-[server|client|relay] etc)
> 
> I look forward to being on the same page and to understand where this is
> going, the likely/potential impact; the naming conventions; etc.

I hope I anwered properly to your expectations, if not please to not hesitate to
ask more :)

Best regards,
Bapt
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://lists.freebsd.org/pipermail/freebsd-ports/attachments/20161222/95e338e6/attachment.sig>