Re: NPM importer

[swedebugia]

On 2018-11-21 23:01, Brett Gilio wrote:
> Mike Gerwitz writes:
>
> > The JavaScript community has poor licensing practices, and the culture
> > is somewhat hostile to the ideals of the free software movement (they
> > focus on permissive licensing to empower non-free software developers
> > using those libraries).
>
> To say the least. It will take a good deal of implementing a license
> checker on the importer, as well as human verification to ensure that we
> are maintaining a high ethical standard.

We might want to use the same approach as licensee:

"The solution
Licensee automates the process of reading LICENSE files and compares 
their contents to known licenses using a several strategies (which we 
call "Matchers"). It attempts to determine a project's license in the 
following order:

If the license file has an explicit copyright notice, and nothing more 
(e.g., Copyright (c) 2015 Ben Balter), we'll assume the author intends 
to retain all rights, and thus the project isn't licensed.
If the license is an exact match to a known license. If we strip away 
whitespace and copyright notice, we might get lucky, and direct string 
comparison in Ruby is cheap.
If we still can't match the license, we use a fancy math thing called 
the Sørensen–Dice coefficient, which is really good at calculating the 
similarity between two strings. By calculating the percent changed from 
the known license to the license file, you can tell, e.g., that a given 
license is 95% similar to the MIT license, that 5% likely representing 
legally insignificant changes to the license text."
https://github.com/benbalter/licensee

We could perhaps also semi-automate the generation of emails to authors 
of the offending npm packages with unclear packages.
Say only 1% of 470.000 has unclear license, that equals 4700 emails to 
authors. :)

In a hypothetical scenario with import of 20 npm packages a day it will 
take us 477.000/20 = 23850 days = 65 years to import them all.

In a hypothetical scenario with import of 500 npm packages a day it will 
take us 477.000/500 = 954 days = 2,6 years to import them all.

This is based on the assumption that all are free software, but that is 
probably not the case.

BTW
ssb-patchwork had over 400 dependants in 10+ levels and the dotfile is 
attached. The rendered png is crazy looking. Reminds me of the holy 
spaghetti monster.

A graph of all npm packages and top packages is also available: 
https://exploring-data.com/info/npm-packages-dependencies/
--
Cheers
Swedebugia

ssb-patchwork.dot.bz2
Description: application/bzip