Re: Packaging problems

[Martin Schanzenbach]

On Thu, 2022-06-02 at 16:29 +0100, Willow Liquorice wrote:
> Right. Perhaps the onus is on the developers (i.e. us) to make things
> a 
> bit easier, then?


Well, the reason why the "lax" distros have outdated packages is
exactly because of that. Somebody decided to contribute once or twice
and then dropped the package.
This would be less likely in less lax-ly managed distros I would
assume.

> 
> To be honest, I barely understand how the GNUnet project is put
> together 
> on a source code level, let alone how packaging is done. One of the 
> things I'm going to do with the Sphinx docs is provide a high-level 
> overview of how the main repo is structured.

I think I am going to keep this copr semi up-to-date from now on:
https://copr.fedorainfracloud.org/coprs/schanzen/gnunet/

This can serve as a basis on how GNUnet could be packaged.
But, packaging is very specific to distros. It really does not make
sense for us to do that for any tiny little distro.

Note that functional composition of GNUnet and packaging are not
necessarily the same.
For example, it makes sense to make mariadb/postgres plugins optional
so that dependencies do not explode further. But why make tiny packages
for the components of GNUnet? You may as well just enable/disable what
you do not need from the installed package.
The package/binaries are not that big (~15MB installed from the RPM) so
size cannot be the argument.
Severely constrained OSes/HW need customized packages, yes, but those
devs know better than we do on what is needed.

> 
> On the subject of complexity, I attempted to disentangle that awful 
> internal dependency graph a while ago, to get a better idea of how 
> GNUnet works. I noticed that it's possible to divide the subsystems
> up 
> into closely-related groups:
>         * a "backbone" (CADET, DHT, CORE, and friends),
>         * a VPN suite,
>         * a GNS suite,
>         * and a set operations suite (SET, SETI, SETU).
> 
> A bunch of smaller "application layer" things (psyc+social+secushare,
> conversation, fs, secretsharing+consensus+voting) then rest on top of
> one or more of those suites.
> 
> I seem to recall that breaking up the main repo has been discussed 
> before, and I think it got nowhere because no agreement was reached
> on 
> where the breaks should be made. My position is that those 
> "applications" (which, IIRC, are in various states of "barely 
> maintained") should be moved to their own repos, and the main repo be
> broken up into those four software suites.
> 
> As Maxime says, GNUnet takes a long time to compile (when it actually
> does - I'm having problems with that right now), and presumably quite
> a 
> while to test too. The obvious way to reduce those times is to simply
> *reduce the amount of code being compiled and tested*. Breaking up
> the 
> big repo would achieve that quite nicely.

It really does not (on modern hardware).
See:
https://copr.fedorainfracloud.org/coprs/schanzen/gnunet/build/4501586/

It takes around 7mins to install & compile from scratch (this includes
installing all dependencies!).

IMO right now "make check" is kind of annoying because it takes too
long and fails because of bad test design. It needs some love.
Maybe a high-level, quick "make check" and an optional "make check-
thorough" idk.

> 
> More specifically related to packaging, would it be a good idea to
> look 
> into CD (Continuous Delivery) to complement our current CI setup? It 
> could make things easier on package maintainers. Looks like Debian
> has a 
> CI system we might be able to make use of, and all we'd need to do is
> point out the test suite in the package that goes to the Debian
> archive.

We build tar.gz every night: https://buildbot.gnunet.org/artifacts/

Packages can be built from that.
But, it does not really make sense to integrate distro packaging with
our releases ("CD"):
Packages have a disjunct release and update cycle.
Usually #packageReleases >= #upstreamReleases.

My 2 cents

signature.asc
Description: This is a digitally signed message part