[BRAINSTORMING] simplifying maintainer's life

[Bryan Drewery]

On 9/3/2014 10:00 AM, Baptiste Daroussin wrote:
> On Wed, Sep 03, 2014 at 04:56:22PM +0200, Tijl Coosemans wrote:
>> On Wed, 3 Sep 2014 15:50:30 +0200 Baptiste Daroussin <bapt at FreeBSD.org> wrote:
>>> On Wed, Sep 03, 2014 at 02:56:14PM +0200, Tijl Coosemans wrote:
>>>> On Wed, 3 Sep 2014 10:25:39 +0200 Baptiste Daroussin <bapt at FreeBSD.org> wrote:
>>>>> On of the most borring thing IMHO in the plist maintainance is all the
>>>>> directories.
>>>>
>>>> Another idea is to support shell glob patterns (*?[) in pkg-plist.  This
>>>> is possible now thanks to staging.  It would allow moving PORTDOCS,
>>>> PORTDATA and PORTEXAMPLES to pkg-plist.  But more importantly, it would
>>>> allow automatic plists that some ports create in post-install to be
>>>> turned back into a real pkg-plist.  Without glob patterns some pkg-plists
>>>> are just too long or too complicated depending on options.
>>>
>>> We could also say pack everything that is in that stage directory.
>>>
>>> The problem is right now I do like static plist because if something
>>> fails to build for $reason, that the plist doesn't find a file in the
>>> staging area and we notice quite quickly that something as failed.
>>> with autoplist or globbing we do lose this feature and we need to way
>>> deal with it.
>>> In anycase we won't make full autoplist because we still need to be
>>> able to specify credentials files per files if needed. But glob is
>>> really appealing :)
>>
>> I completely agree :)  Files should be listed explicitly if possible,
>> but sometimes it's very inconvenient and in these cases some ports
>> roll their own autoplist implementations which worse than having a
>> static pkg-plist with a few glob patterns.
>>
>> Moving PORTDOCS etc. to pkg-plist means all package content is listed
>> in one file.  That will probably simplify check-plist too.
> 
> Right I'll implement globbing (one level not recursive)
> 
> regards,
> Bapt
> 

I disagree globbing simplifies check-plist. I see it making it much more
complicated.

I support autoplist. The point that if something failed to build and is
now missing for packaging, yet expected, exists today already. I never
have carefully analyzed the build of a port to see if it is building
everything I expected down to every file. I trust that if there is an
error building something I requested that it will halt the build. Does
this make me a bad maintainer? Probably. I'm sure most maintainers do
this. It builds, it works, ship it. The "it works" part is obviously
skipped by many as well. makeplist now assumes the build is proper, so
I'm sure many maintainers do the same. Whatever makeplist (or tinderbox
or poudriere the past X years) has been telling them. I've seen only a
handful of cases where something didn't build but the build itself
passed and hit the plist error. The vast majority of builds do not have
this problem.

If there is really a file that you are worried won't build, put in a
check in post-build to fail if it is missing.

Only 1-2 year ago when majority of users were using ports, and not
packages, we had the "missing file from build" problem even more so and
we survived. When the port just straight installed to /usr/local,
nothing did validation of files installed. Evidence for this was when
pkg DEVELOPER_MODE came in and we started using poudriere quite often
and finding that many ports advertised some file in their plist that did
not really exist.

If you add globbing, you will get someone using '*' as their entire plist.

I strongly believe that autoplist should be the norm for most ports. At
work I have been implementing it based on 'make makeplist', and having
it auto add in appropriate keywords (not yet upstreamed). This then only
requires ensuring DOCSDIR, EXAMPLESDIR are properly set in the Makefile.

What might be interesting is an autoplist with glob-like syntax to
support sub-packages. So a main package could be everything except
.debug, or .h, or man/*. Or each of those prefixed with a keyword
specifying with package they are in.

I work at an embedded vendor and it would be hugely helpful to be able
to easily move symbols, headers, manpages, examples and docs all into 1
or more sub-packages. Doing all of this from bsd.port.mk, optionally, is
even better.

I really dislike plists as they are today. The argument that having them
makes it easy to see what the port installs is bogus. Many don't have
pkg-plist because they have PLIST_* or INFO or are missing DOCS/EXAMPLES
due to usage of *. Many, such as rubygems and python, have autoplist
already. I would like to provide 1 approved way to do autoplist, rather
than having the dozens of different implementations that we have today
in ports.

If you want to see what a package installs, look at the package
repository. Pkg has a means to advertise this and we should enable and
export it in an efficient way. Portsmon, or Freshports, could
potentially feed off this database and display it on their own sites as
well.

For non-file plist entries such as @exec, these should be in
install/upgrade/deinstall scripts. The plist is already a bad place for
them. We've been inlining more and more of this and it is far less
readable and maintainable in many cases.

You have to remember that plist was only born because there was no
stagedir (or filemon or install -M). We have a stagedir now. Plist is
redundant. Plist maintenance is just busywork for maintainers and
portmgr. Orphaned files, leftover files, listed-but-missing,
mtree-owned, dependency-owned. All of these go away if we use autoplist.
Maintaining ports becomes simpler. Note that I do not propose requiring
autoplist, only providing it. If some port has a case where autoplist
does not make sense then it does not need to use it.

I understand there is fear involved with not having a plist validate
everything. Consider that many other package systems do not require a
plist to start.

-- 
Regards,
Bryan Drewery

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: OpenPGP digital signature
URL: <http://lists.freebsd.org/pipermail/freebsd-ports/attachments/20140903/20b5493e/attachment.sig>