Re: Octave Forge -- Looking for a new leader

[Oliver Heimlich]

On 02.01.2017 12:58, Philip Nienhuis wrote:
> Olaf Till wrote:
>> On Sun, Jan 01, 2017 at 03:27:31PM -0800, PhilipNienhuis wrote:
>>> Sebastian Schöps wrote
>>>>
>>>> Olaf Till-2 wrote
>>>>> On Sat, Dec 31, 2016 at 01:40:05AM -0800, Sebastian Schöps wrote:
>>>>>> However, I still believe that we should give up on the idea of
>>>>>> supported
>>>>>> and
>>>>>> unsupported packages and just maintain a list of known packages
>>>>>> (hosted
>>>>>> on
>>>>>> 3rd party sites) that are compatible (i.e. have a Makefile like you
>>>>>> suggested recently).
>>>> I'd say that official packages must be carefully reviewed such that
>>>> there
>>>> is no malicious code and that the code is (mathematically) correctly
>>>> working. I understand that Carnë checked formal correctness but not
>>>> contentwise (Carnë: right?). I think it would be more honest to
>>>> maintain a
>>>> list and make packages less part of the Octave project itself. This
>>>> would
>>>> also further reduce the effort for reviewers and submitters. Many
>>>> packages
>>>> are doing their development anyhow somewhere external.

What do you mean by “(mathematically) correctly working”?  Would you say
that we can achieve peer reviewed package content?  Or is it just about
checking whether enough tests exist and pass?


>>>> Of course, also for Octave there is no rigorius guarantee that all
>>>> functions give the correct answer, nonetheless the effort that peope
>>>> invest to ensure correctness is obviously much higher than for
>>>> packages.
>>>> This is rather obvious since many packages require very specialized
>>>> knowledge, e.g., I can check rather easily if the pcg implementation is
>>>> correct but I have no clue how the algorithms of the interval
>>>> package work
>>>> and it would take an incredible amout of my time to do a code review.
>>>
>>> There's indeed more to it than just "working mathematically correctly".
>>>
>>> Several OF packages interface to external software and data
>>> structures/data
>>> files. Notable examples are the io and the mapping packages that I
>>> maintain.
>>> I really don't expect an OF "leader" to be able to effectively review
>>> such
>>> code. It is just too specialized. As regards quality in the sense of
>>> "correctness" the best is to hope for code maturity, usually obtained by
>>> prolonged use in the real world and fixing the bugs encountered there.
>>>
>>> As far as quality is concerned all that an OF leader can do is review
>>> package structure, check completeness of package documentation (easily
>>> overlooked by package maintainers) and maybe stimulate tests covering
>>> the
>>> complete package; all of this is very valuable in itself.
>>> Add some cursory code style checks to the job and he/she runs out of
>>> time
>>> ....
>>
>> Something more that can be cared for with 'supported' packages
>> (keeping the 'supported' notion):
>>
>> - package licensing
>>
>> - (some) consistency of contained functions with Octave, with other
>>    packages, with ML (has been disputed, can require compromises from
>>    package maintainers)
>>
>> - excluding unmaintained packages
> 
> Thanks Olaf,
> 
> These good additions clearly help illustrate that being OF leader is
> quite a voluminous job.
> 
> Would it be good to put this list of responsibilities somewhere on the
> wiki, or in the admin section of Octave-Forge, rather than leave it
> buried in this mail archive?
> 
> 
> It may be useful to list the responsibilities of maintainers of
> (supported-) packages as well, to help distinguish who has what
> responsibilities:
> 
> - Keep packages in good shape (a wide description, yep):
> "installability", up-to-date INDEX / DECRIPTION / NEWS files,
> PKG_ADD/PKG_DEL things and autotools stuff for binary functions;
> 
> - Review contributed additions (incl. bug fixes) to the package for
> coding style, help texts, tests, obvious logic and coding errors.
> 
> - .. any more ...?

I have compiled a list of roles and responsibilities in the wiki, which
should reflect the current state (where Carnë represents the two roles
“package manager” and “leader”).  Please add things which I might have
overlooked.

http://wiki.octave.org/Octave-Forge

I believe that the time consuming task of processing the release tickets
can easily be distributed among several people if the quality criteria
for package releases can be agreed upon.  I can help with that task
myself (won't be available 24/7, but you can count on my support).

> Revisiting Sebastian's valid point about correctness of content, that
> looks to be a debatable responsibility even for package maintainers, let
> alone OF leader(s). Do package maintainers really need to know all
> nitty-gritty of functions contained in the package?
> Taking the io package as example, it contains contributed and inherited
> functions dealing with e.g., json and pch formats I am completely
> unfamiliar with. I could only check help texts and conspicuous coding
> style issues etc. but not what the functions really do.

IMHO, in the long run we have to push more tasks from the Forge team to
the package maintainers and from the package maintainers to authors and
contributors.  Otherwise Octave Forge cannot scale with more and more
packages. For example:
 * package maintainers should be able to actually make the release on
their own
 * the forge team can add it to the list, if quality criteria match

> Question then is: *who* is responsible at all for correctness of
> functions in OF packages?

## This program is distributed in the hope that it will be useful,
## but WITHOUT ANY WARRANTY; without even the implied warranty of
## MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
## GNU General Public License for more details.

If the package maintainer gets to know about actual errors, she should
work towards fixing it (not necessarily herself).  Eventually she may
decide to remove a problematic function from a package.

Oliver