The ports collection has some serious issues

Thu Dec 8 12:28:06 UTC 2016

On Thu, Dec 08, 2016 at 05:16:24AM +0000, Daniil Berendeev wrote:
> Hello guys!
> 
> First of all, it's not a hate mail, I appreciate all the work done on
> the system and I enjoy using FreeBSD every day.
> 
> But after some recent experience I'd like to point out some problems
> that make using the ports collection uncomfortable and painful.
> 
> Some overview before we start:
> * Why I use ports over pkg?
> Because, generally, packages are built with poor default options, for
> example moc isn't able to play .alac/.mod and that's frustrating.

Lot's of work has been done over the last years improve the default options for
general pupose cases. Have you open an issue about that one?

> 
> * Why pkg is still nice?
> It is able to update packages with broken ABI, it's fast and easy to
> use. Some packages/ports don't have options and can be used via pkg by a
> ports user.
> 
> I want to contribute to FreeBSD development, so, long story short, I've
> decided to move to -CURRENT. Everything went fine except the ports upgrade.
> 
> Is it possible to upgrade the ports by hand? Well, it is, but it is not
> too comfortable. Ports collection by itself doesn't provide a nice way
> to work with port management, so a user needs to use something for port
> management. As the handbook advised, I picked portmaster.
> 
> And here begin the problems.
> 
> 1) portmaster is not nice for the user.
> If it comes over an error even in one little tiny port that is a
> dependency for something bigger , it will abort its work and leave all
> the other ports not updated. So, if you try to to do `portmaster -af`,
> you should not forget `-m DISABLE_VULNERABILITIES=yes` (we will return
> to this one later) and you must pray to God for not coming around a
> circular dependency or some port that would fail to deinstall its older
> version. You can't leave portmaster for a night to update all the needed
> ports and deal with broken ones in the morning, you need to cherry pick
> the broken ports and ignore them, and then try to deal with them.
> 
> Although portmaster is not releated to the FreeBSD project and is an
> outside tool, there aren't any alternatives from the project itself. So
> use it or die. Not a nice situation.
> 
> 2) pkg and ports are not in sync.
> pkg appeals to build ports that are from 2xxxQx branches. The promoted
> tool for syncing ports (portsnap) always fetches from head. And there is
> no way to choose. That gives us the next problem:
> 
> 3) no integration between ports and packages
> There is no clear, easy way to use ports and packages simultaneously. If
> I'd like to use some built packages to speed up port updates, I have to
> ignore by hand all the packages that I want to be built as ports. It's
> easier to stick to only ports or only packages.
> 
> 4) uncomfortable way of rollback
> If I want to rollback, or just choose the branch from where the packages
> are built (to stay in sync with pkg), I have to pull the whole svn
> repository.
> 
> 5) svn repository.
> I don't want to spark a holy war and I don't belong to those type of
> people who are always obsessed that something isn't done in their way.
> But guys, svn is not a good tool for ports. Just for one reason,
> actually (as for me, I could tolerate anything else, but not this one)
> -- size. The size of repository is 20G+ and growing. I don't want to
> pull 20G+ in /usr/ports just because I need to use ports. It's just
> sick. The repository is so big because, as all ya know, svn is expensive
> in branch operations. Since you've began to do those 2xxxQx branches the
> size of the repository began to grow rapidly. It's inefficient and
> uncomfortable. For such a work something like git or mercurial should be
> used, they'd fit in 3-4G.
> 
> 6) broken ports are pushed to head
> Why do we have such a situation, when head contains a handful of broken
> ports? Why commit a port that won't build? It's sick.
> Ports are broken in a different way. Some fail to build. Some fail to
> uninstall their older version (like rust), so that you need to do
> `pkg remove -f portname; portmaster portname`. Some have a circular
> dependency (d-bus) and will try build until the heat death of the
> universe. I just don't get it, why broken ports are pushed to head, if
> head is then used by portsnap to update /usr/ports? You leave tons of
> users with a broken setup. And there is always a bunch of ports that
> won't build. It's not just one, or two, it's a handful of ports.
> pkg-fallout at FreeBSD.org is overwhelmed with build fails.
> 
> 7) No way to update ports with broken ABI.
> I need to run `pkg update` and then pick the broken ports by hand. Or do
> `portmaster -af`.
> 
> 8) ports with vulnerabilities.
> They exist in the tree and on build attempt they shout that they won't
> build without DISABLE_VULNERABILITIES=yes. The catch is that there is
> always a bunch of ports with vulnerabilities. So if you are doing a
> fresh install, you have to install those nasty vulnerable ports anyways.
> It causes you to do extra moves and doesn't add no security or safety.
> There is no way to pick the latest safe version.
> 
> I hope that my mail will produce a productive discussion that will lead
> to some good decisions for fixing these problems.
> 

Have you considered using things like poudriere that would allow you to build
your own repository with your own set of packages and options.

You will benefit:
- ability to use pkg for your upgrades
- ability to use customize your packages
- safe rebuild process (in case of broken ABI)

Best regards,
Bapt
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://lists.freebsd.org/pipermail/freebsd-ports/attachments/20161208/bd443432/attachment.sig>