svn commit: r440978 - in head: devel devel/npm-amdefine devel/npm-clean-css devel/npm-clean-css/files devel/npm-commander devel/npm-commander/files devel/npm-graceful-readlink devel/npm-source-map ...

[Adam Weinberger]

> On 17 May, 2017, at 8:49, David Naylor <naylor.b.david at gmail.com> wrote:
> 
> On Tuesday, 16 May 2017 09:23:13 Adam Weinberger wrote:
>>> On 16 May, 2017, at 2:57, Rodrigo Osorio <rodrigo at osorio.me> wrote:
>>> On 05/16/17 00:27, Adam Weinberger wrote:
>>>>> On 15 May, 2017, at 16:05, Rodrigo Osorio <rodrigo at FreeBSD.org> wrote:
>>>> What do all these ports do that npm doesn't already do, especially given
>>>> that this will cause breakage for everybody with any of these modules
>>>> installed globally?
>>>> 
>>>> # Adam
>>> 
>>> Hi Adam,
>>> 
>>> That's npm dependencies that need to be packages properly, I'll feel
>>> unsafe if the port relies on npm to download external sources from github
>>> during the packaging, without checking the integrity.
>>> 
>>> Of course, this could be an issue for peoples who deploys npm packages on
>>> their own, but how can we handle this ? Maybe dep search path should be
>>> set manually if you install unpackaged npm tools ?
>>> 
>>> I run a quick test, but apparently node uses in tree dependencies before
>>> global.
>>> 
>>> Cheers
>>> -- rodrigo
>> 
>> The point is that clean-css is already installable by "npm install
>> clean-css" or "npm install -g clean-css". The reason there were no node
>> modules in the ports tree is that npm handles node modules better, manages
>> dependencies automatically, and doesn't break things.
>> 
>> My gut feeling is that these ports are not a good idea. There's currently no
>> policy about node modules, so you didn't do anything wrong by adding them,
>> but I don't think these ports are beneficial.
> 
> I think this covers a broader topic of should we "port" other "packages"
> of software provided through an external mechanism, such as:
> 1) pip (Python)
> 2) nuget (.NET)
> 3) cran (R)
> 4) npm (Node.js)
> 
> All of the above provide there own ways to manage dependencies, to a greater 
> or lesser extent.  
> 
> I think there are three motivators for including these "packages" as part of 
> Ports:
> 
> - Binaries: some of these "packages" require compiling, and thus need to be 
> kept consistent with the broader ecosystem (i.e. Ports).  Keeping these 
> "packages" as part of Ports will ensure consistency with the rest of the 
> system's binaries.
> 
> - Dependency: a "package" may be a dependency on a i) a binary "package", or 
> ii) a program (i.e. some other port).  The dependant could either include the 
> dependency, privately, or the dependency needs to move into Ports.  The latter 
> is obviously better from a consistency and maintainability perspective.  
> 
> So, I advocate for the inclusion of these "packages" in Ports.  However, if 
> pkg ever develops the ability to "bridge" with these other frameworks then an 
> alternative approach could be warranted.  

Your reasoning is solid, David, except that node (and Go) modules are supposed to behave in a way that the FreeBSD ports system doesn't handle.

There are two general paradigms for language-specific package management. There's the old-style way, used by Perl and Python, where one version of a module is installed globally, and the expectation is that all modules will use whatever versions of other modules are installed. Those types of modules make good candidates for ports, because the ports paradigm works in the same way.

A newer paradigm, used by node and Go, is for each module to bundle its own dependencies in an installation. Dependencies of a module are essentially private to that module, and so each module may have exactly the version of each dependency that it needs, and they stay private to that module. This is not how FreeBSD packages behave, and it abuses both the FreeBSD paradigm and the npm/Go paradigm to make one behave like the other. Furthermore, npm often expects to alter installed modules depending on the environment.

As for npm, it actually breaks when things are installed globally. Once modules are installed globally by ports, npm can no longer update or modify any modules installed globally by npm. I know absolutely nothing about Go, so I can't speak to specifics, but swills sent a detailed explanation of it a couple weeks back.

For these reasons, node modules and Go modules simply are not good candidates for FreeBSD ports. npm manages node modules in the correct node way (and pkg doesn't), and so npm is the correct way to install node modules. I'm sure Go has a similar tool; I'll call it gopher because that sounds like a good name for a Go installer, and I'm sure that it installs Go modules the correct way, and so it is the correct way to install Go modules.

There are definitely exceptions to this: if a module must be modified in a specific way to work on FreeBSD, or if a module is a required dependency of another port, then having those modules in the tree definitely makes sense.

But past that, node and Go modules shouldn't wind up in the ports tree. Their package managers handle dependencies the correct way for those languages, and FreeBSD ports doesn't. The PHB doesn't talk about it yet, but I'm working on a writeup for that too. In the meantime, I've asked Rodrigo to back out those node modules (and AFAIK the Go modules are headed in the same direction too).

# Adam


-- 
Adam Weinberger
adamw at adamw.org
https://www.adamw.org