Re: Alternative to Source Forge for Octave Packages (Was : Re: pdepe)

[Oliver Heimlich]

On 31.05.2016 12:49, Carlo De Falco wrote:
> 
> On 31 May 2016, at 12:21, Oliver Heimlich <address@hidden> wrote:
> 
>> SourceForge is behaving very well under the new ownership so far. Also I
>> have the impression that the Julia community is in a much bigger vendor
>> lock-in compared to us due to their deep integration of GitHub (and no
>> other providers) and Travis CI. Both third-party proprietary services.
>> They have waived control over their infrastructure.
> 
> I think we don't want to keep full direct control of our infrastructure 
> either.
> 
> We just don't want to be (again) so tightly integrated with one particular
> vendor that we cannot easily switch in the future.
> 
> Ideally I would love to have each package hosted independently
> and only the metadata repository (and maybe point release tarballs) 
> centralized.

That definitely is a great target. To be able to self-host a package and
SCM if one wants to, or—more likely—to change service providers as the
maintainer sees fit. One risk that it bears is that project websites
might vanish. However, you would still have the point release tarballs
centralized (or stored at an archive service like Zenodo or even the
project history as we currently have on Octave Forge).

>> As far as I see it the advantages over Octave Forge are:
>> - higher automation
>> - continuous integration for all packages
>> - a distributed(?) review process for package uploads (which scales
>> better than a single admin)
>> - an option for the user to install development versions
>> - a cleaner separation of the metadata repository, although they don't
>> make much use of it if everything is GitHub only.

Also there should be TLS secured downloads. (SourceForge is already
working in that direction.)

> Sounds like a good list of advantages already, but you are probably
> right that we need to improve on their model if we want a system
> that really suits our needs.

One more thing that should be noted is that the Pkg command in Julia
also manages the package release process (we currently do this with
Makefiles in some Octave packages) and can do a lot of checks
automatically. It adds uniformity/consistency between packages.

> It would be great if we could get a proof-of-concept demonstrator
> in time for OctConf in Geneva this autumn ...

There are some big questions to start with: Do we need automated QA
infrastructure? Can this be distributed as well (e. g. each package uses
different service of choice: Hydra, Jenkins, Bamboo, Travis CI,  …)?
Where shall the point releases be hosted? Can this be distributed and or
mirrored as well? How will the metadata repository be organized (and by
whom)? What are the minimum standards that a package has to fulfill?

Everything else will be a question of proper implementation, tooling
support and convenience features. The tooling in Julia does everything
to simplify creation and publication of one's own package. If this
target would be missed, you would hardly see many packages in the
repository.

On the other hand I'd like to raise the question if the effort to reach
all that is worth the trouble? How many extra Octave packages do you
expect to see because of a convenient, open, distributed repository?
What value is added in contrast to either self-host a particular package
or use a centralized repository that we currently have with Octave
Forge? Is it really required to have the QA, release archive and
automated installation features or would it suffice to have a public
dictionary of Octave packages with URLs to their actual websites? The
latter might be a valid option if we had a particular file extension for
Octave packages, which can be easily installed from the browser.

Oliver