ports structure and improvement suggestions

Jeremy Chadwick freebsd at jdc.parodius.com
Tue May 9 13:56:43 UTC 2006

On Tue, May 09, 2006 at 09:07:08AM -0400, Robert Huff wrote:
> Jeremy Chadwick writes:
> >  What I proposed in the aforementioned thread was to have an
> >  actual BIND-like configuration file (e.g. a tree) which contains
> >  individual options which are set to yes/no per port.  There is a
> >  global scope which applies to all ports, while individual ports
> >  can be adjusted individually -- or overwride the global scope
> >  settings.
> 	I'm confused: how is this different in principle from the
> "MAKE_ARGS" section of (/usr/local/etc/)pkgtools.conf?  E.g.:
>  MAKE_ARGS = {
>     'www/mozilla-devel' => 'BUILD_OFFICIAL=1',
>     'graphics/gimp' => [
>          'WITH_PYTHON=yes',
>          'WITH_HTML_HELP_BROWSER=yes'],
>     'www/apache22' => [
>          '-DWITH_LDAP_MODULES',
>          '-DWITH_MISC_MODULES',
>          '-DWITH_CACHE_MODULES',
>          '-DWITH_SSL_MODULES'], 
>     'editors/xemacs-devel-mule' => [
>          '-DDISABLE_CONFLICTS'],
>     'editors/emacs' => [
>          '-DDISABLE_CONFLICTS'],
>    'multiledia/mplayer' => [
>          'WITH_GTK1=yes',
>          'WITH_REALPLAYER=yes',
>          'WITH_OPTIMIZED_CFLAGS=yes',
>          'WITHOUT_MENCODER=yes'],

The configuration syntax (my recommendation vs. pkgtools.conf) are
near-identical.  Syntax-wise, they do essentially the same thing, or
amount to the same thing.  I don't even need to know details to know
the above is (probably) being read into the interpreted language
directly (ex. for perl, using `require`).  It's a hash of arrays,
and the syntax is for programmers.

The pkgtools.conf syntax is a nightmare for non-programmers.  Forget
a [ or ], forget a comma, forget to single-quote the key or the value
or both, and the administrator begins The Quest(tm) to figure out where
the problem really is.  This is because the file isn't parsed by the
application -- it's being parsed by the interpreter as code.  Make a
typo and the end user says "Uhh, what?" and files a PR, only to find
they forgot a comma.  End users need to learn file formats and syntax,
yes, but they don't need to be spending their time visually parsing
what you see above.

Anyways.  The points I'm trying to make here are this:

* Tool in question _must not_ rely on any third-party software or
  or interpreters.  Every piece must come with the base system.
  That means no perl, no ruby, no python, no tcl.  This has been a
  rule of the FreeBSD ports system since the release of RELENG_5
  and the removal of perl in the base system (which I still think
  was a *fantastic* decision).

* MUST NOT have its "own database" for dependencies or relational
  information.  Plain-text/flat files are fine, or using UNIX
  directory structures (or symlinks, blah blah) are fine too.

  For example: portupgrade has its own Berkeley DB file which it
  considers "superior" to /var/db/ports.  I cannot count how many
  times I've seen administrators fighting with the fact that their
  portupgrade DB "is not in sync" with the actual base system.
  I want to avoid this situation at all costs.  I've been told that
  the reason portupgrade uses its own Berkeley DB is because "of
  flaws/oversights in the current base system ports implementation".
  If this is true, what are they, and how can we fix those at the

* Configuration file must be parsed, not "read directly into the
  language" (e.g. no using `source` :-) ).  This is to ensure the
  tool spits out human-readable errors when the parser fails,
  yadda yadda.  I don't want to see 10 lines from the language
  interpreter bitching about what amounts to a missing comma -- I
  want to see a single or pair of lines to stderr stating there's
  a parser error on line 6.

* Must integrate with the ports system -- somehow.  I don't know
  if the tool would require custom patches to make(1) or if it
  could be added to the ports/Mk framework.  I do not know much
  of the "magical" sh internals in Ports/Mk (never sat down to look
  at them).

  Could this implementation be solved by using a pre-parser (byacc,
  etc.) somehow?  I'm only familiar with m4, and m4 does not seem to
  be the tool for the job here, but byacc looks like it could help,
  especially in regards to the config parser.

Also, big thanks to Sideris Michael for bringing all of this up --
this is a conversation I think many of us who use ports as well as
those who maintain ports (I fall into both categories) have been
dreading/avoiding for a long time.

It's good to get ideas / conversations going about such influential
things.  :-)

| Jeremy Chadwick                                 jdc at parodius.com |
| Parodius Networking                        http://www.parodius.com/ |
| UNIX Systems Administrator                   Mountain View, CA, USA |
| Making life hard for others since 1977.                             |

More information about the freebsd-ports mailing list