Python 2.7 removal outline

Thu Mar 25 15:03:25 UTC 2021

On Thu, Mar 25, 2021 at 12:02:29PM +0100, Olivier Certner wrote:
> Hi,
> 
> Maintainer of Tauthon here, and of Pale Moon (for the few hours it lived in 
> the tree in February; but I'm still pushing updates to PR 251117).
> 
> I find this announcement very much disappointing, because the situation for 
> ports that need Python 2.7 or similar to build doesn't seem to have changed at 
> all. In short, we are just told (again) that they should disappear.
> 
> But now we are told also that there are exceptions to the general rule. And 
> even worse (or better, for the comical effect), that the Chromium exception is 
> going to force its agenda on portmgr@, because nobody seems to really know 
> when that migration will complete (having quickly studied how it worked out 
> for Firefox, I'm willing to bet this will never happen before June, and likely 
> will not before the end of this year either). "I love it when a plan comes 
> together."
> 
> I had hoped that portmgr@ would turn to me (and others in the same situation) 
> with at least some way out to allow Pale Moon to go into the ports tree. In 
> its mail, the closest thing I can find is this:
> > - you are of course free to provide your own version of Python 2.7, Tauthon
> >   and any application using those languages in your local setup, by using
> >   overlays for example.
> So, if I understand correctly, in its great magnanimity, portmgr@ allows us to 
> do what we want locally. Gosh! That's news. Didn't know I had been violating 
> the law for the past 10 years by doing exactly that with my set of patches to 
> the ports tree (a long way before overlays even existed; that said, I think 
> overlays are helpful, more on that below).
> 
> And there's more in the same "tone":
> > No excuses. * 3
> You probably were not addressing this to me in particular (at least I hope). 
> But just in case, did you mean: Excuses for not respecting a non-existing 
> policy?
> 
> In December last year, I submitted Tauthon with the goals to be able to use it 
> to build Pale Moon and help others that needed to run 2.7 code despite phasing 
> out of PSF's interpreter. It was pushed on 2020/12/11. I then submitted an 
> upstream-approved official version of Pale Moon port, that was pushed on 
> 2021/02/15. It was removed less than 4 hours later, together with Tauthon, in 
> "rage commit" 565350 by antoine@, without any public (or private to me) 
> communication, before or after. Except for one thing: He responsed to my 
> request for explanations by saying: "When we deprecate python 2.7, we also 
> deprecate all forks of python 2.7.".
> 
> That also was news to me. Don't know how many ports would have to be removed 
> if this rule was followed for all forks of now deprecated projects. I don't 
> think there would be only a few of them. What about the fact that Tauthon has 
> lived 2 months in the tree up to the removal? What about that it was 
> contributed through a public PR, linked to other Python-related PRs by some 
> committers? What about that nobody issued a public objection to its presence 
> on said PR or on mailing lists? What about that it is still listed *today* on 
> the wiki's WantedPorts page? Never had a single answer on all this. Surely 
> these must have been signs that what I was doing was very wrong indeed, and 
> against a very clear and strong policy, which I should have understood by 
> myself. Idiot me.
> 
> I'm not going to repeat here the long mail I wrote in response to FUD being 
> spread about Pale Moon (and even Tauthon and Python 2.7 to some extent) by 
> rene@ and danfe@, and its excruciating details on how deluded they were 
> (still are?). These mails were for committers only, and exchanged on 
> 2021/02/2{1,2}.
> 
> I'll just quickly point out the new FUD and inaccuracies, add some facts and 
> opinions to the picture, and wrap up with two possible ways out.
> 
> > Tauthon is not guaranteed to be compatible with any official
> >   Python version so keeping it would just unnecessarily complicate things.
> 
> mandree@ preceeded me, so not going to add much except that Tauthon in 
> practice lives up to its promise (there are incompatibilities due to the 
> introduction of some Python 3 features, but they are very minimal and hard to 
> trigger; unfortunately for me, Pale Moon's weird build system, inherited from 
> Mozilla, triggered some).
> 
> As to why it would complicate things, we are left with no clue, especially 
> given that Tauthon is not hooked at all into python.mk and that no port 
> currently depends on it (Pale Moon having been removed).
> 
> > - On a related note, most software using Python 2.7 was already removed
> >   from the Ports Tree last year, a lot of it being unmaintained or
> >   more or less abandoned upstream.
> 
> 1. Most is not all, unfortunately. What happens for the rest? Is "Well, let's 
> pretend it doesn't exist, and shut the contributors up." your response? 
> Seriously?
> 2. Security is a non-issue for build-only dependencies.
> 
> > - We are indeed faster with dropping Python 2.7 than e.g. Ubuntu, however
> >   more recent Debian/Ubuntu distributions are more and more dropping Python
> >   2.7 too. This also has to do with how their branching model works, the
> >   package set of Ubuntu LTS is determined a few months before the release
> >   itself.
> 
> Debian is still tolerating Python 2.7 for build-only dependencies in bullseye, 
> which is due to be released imminently, and will be supported until around 
> 2024. Ubuntu 20.04 LTS incorporates it, apparently without restrictions (I see 
> a full suite of packages relying on 2.7 there), and this release will be 
> supported until... April 2025. So, yes, faster by at least 2 years.
> 
> Surely, we are not organized the same, and do not have the same manpower and/
> or money. However, their security teams do not seem to think that phasing out 
> CPython 2.7 right now is of uttermost importance. Some Debian links on the 
> topic:
> https://tracker.debian.org/pkg/python2.7
> https://wiki.debian.org/Python/2Removal
> I must point out that this last page, although listing interesting links, 
> seems itself seriously outdated, as it is contradicted by facts (e.g., 2.7 is 
> in bullseye, and it is indeed receiving security fixes, see the first link). 
> It seems that they have changed their mind in light of needs and demands. Food 
> for thought for portmgr@?
> 
> And again, there would be no hurry at all for build-only dependencies. Or is 
> there? May I ask on which ground exactly?
> 
> > As can be seen on [2], multiple vulnerabilities already have
> > been fixed for Python 3.6 to 3.9 this year.
> > [2] https://www.python.org/downloads/release/python-392/
> 
> I've started looking into these vulnerabilities. Most are simple to 
> understand, and their patches even readily apply to Tauthon when relevant. 
> Going to submit a bunch of them upstream. At least, this is possible with 
> Tauthon, contrary to CPython 2.7.
> 
> But in the end, I don't think this is really important for the dependent ports 
> issue, since, again, we are talking about build-only dependencies on CPython.
> 
> That was just for the sake of re-establishing a more accurate balance of 
> facts. Given the track record of recent reactions of portmgr@, I'm now not 
> foolish enough to believe that all that precedes is going to have any visible 
> effect on them.
> 
> Now, for the two possible ways out, I'm still having some hope (but frankly 
> not that much).
> 
> 1. Add the infrastructure to have build-only dependencies. I've proposed 
> changes to that end (https://reviews.freebsd.org/D28946). In addition to the 
> comments in the review, bapt@ rightfully pointed out that 'make install' would 
> still be possible to run for ports listed in NORUNTIME. I acknowledge that 
> this is indeed a problem in the current problem, but think it could be solved 
> technically (e.g., forbidding 'make install' for those ports, but allowing it 
> when building a dependent port through an environment variable, and removing 
> the install after the build). Which reinforces my thinking that the "problem", 
> whatever that is, is not technical, but human. Overall, portmgr@ doesn't 
> really seem to be interested in this solution (got short reactions such as 
> "with RESTRICTED, we don't need this", or "this would be a precedent", indeed 
> a useful one if you're asking me).
> 
> 2. Leverage overlays to provide additional repos, a bit like AUR for Arch. 
> Here I'm in fact building on top of one of bapt@'s ideas. Sounds great for 
> publishing ports that are not in the official tree. But not necessarily for 
> package building: I personally won't commit to maintaining a separate build 
> cluster for all arches and supported FreeBSD versions, in the short term at 
> least.

I really think we should as a project move forward to that direction, it does
not even need to be driven by protmgr or even drive by any @freebsd.org

I would argue here that it is even more interesting to go the gentoo way try to
provide a tool to just server as a directory of available overlays
with https://github.com/freebsd/poudriere/pull/798 and
https://github.com/freebsd/poudriere/pull/797 it is even easier to public a
light repo per overlay.
> 
> I re-read portmgr@'s charter (https://www.freebsd.org/portmgr/charter/). I 
> wish it contained points about proper planning, communication and helping 
> maintainers and committers instead of destroying their work without notice, 
> even for "niche" ports. Perhaps it doesn't because this was implicit or taken 
> for granted. In which case, in light of recent events, it may be a good time 
> to revise it.

I will only here answer about the quality of the communication of portmgr, yes
there is room of improvement in general in the current portmgr team as of how we
do communicate about plans and policy and we are working on it.

Best regards,
Bapt
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://lists.freebsd.org/pipermail/freebsd-ports/attachments/20210325/afa54b9c/attachment.sig>