I made a crawler and I let it loose on the jquery 3.1.1 dependencies on
registry.npmjs.comIt recursevely fetched the dependencies of jquery 3.1.1, then the dependencies of the dependencies, then the dependencies of the dependencies of the dependencies... and so on
Until there were no more dependencies to fetch.
It stored all such dependencies in a graph database made by the amazing amz3 for their project, Culturia. They wrote about such project several times on the guile-users mailing list.
It took days to download. And it took me months to produce a working version of this code. Because I don't know Guile very well and maybe I'm not that smart overall.
Anyway, now I have a COMPLETE graph of the dependencies of jquery 3.1.1
It's made of
47311 vertices and
324569 edges
I made a graph of a subset (graphviz chocked a bit on this ;-) )
It's here
http://catonano.altervista.org/grafo.svgThere are 448 "broken" packages. Probably I should explain how I classified packages as "broken". Maybe not in this email.
Anyway, these broken packages pose a challenge to the mission of porting Jquery into Guix, in my opinion, and they should be considered with some attention.
There are 1314 packages with NO dependencies that could be used as starting points in porting Jquery into Guix.
Here's the list
http://catonano.altervista.org/broken_packages.txtIf there's anyone interested, I can give you the data folder so you can try all the queries you want on these data without having to to run this thing for a bunch of hours
In the future, I'd like to run this thing on some other package and merge the graphs so I will be able to investigate which are the common fundamental dependencies for SEVERAL important packages in Nodejs.
So if someone wants to dedicate time to porting Nodejs stuff in Guix they will be able to select most urgent packages to start from.
The same could be said of broken packages taht affect several important packages.
The porting of Nodejs in Guix cannot be done with brute strength. A data oriented approach can help, in my opinion.
The ideal would be to have something that, like bitcoin, coordinates a swarm in such a way that every node can contribute a tiniy bit of data to a common data structure, so all the nodes would have a complete copy of the database.
Collecting a mantaining of datasets should be freed of the client server model too. Not only the social media.
But that's more than I can handle, anyway.
I'd like to talk about the stumbling blocks I run into to discuss Guile and my knowledge of it.
For example, I can't use that thing in the autotools that processes
configure.am files so I just forked amz3's project and added my files in there. As guests. Thanks amz3 !
I'd also like to describe te screw ups in the format I put the data into. I realized my mistakes when hours of crunching had already been done.
They can be migrated to a better format though
If you don't mind, I will discuss these issue in the future, not now.
One last fun fact: while I was watching the output flowing in my terminal, I saw a package called
"broccoli-funnel"
Ok, that's all for now.
