Re: clear policy discussions

[David Kastrup]

Graham Percival <address@hidden> writes:

> On Tue, Jul 10, 2012 at 01:21:38PM +0200, David Kastrup wrote:
>> I mean, we have too few categories for regressions.  I can see the
>> following:
> -snip-
>> My personal opinion is that
>
> My personal opinion is that we should not discuss major policy
> changes in the middle of a thread about dots and staff sizes.

It was in the course of changing the status to "Critical".

> There is a proposal to change the stable release policy:
> http://lists.gnu.org/archive/html/lilypond-devel/2012-06/msg00462.html
> http://lilypond.org/~graham/gop/gop_3.html
>
> That proposal is currently "under debate", although there was such
> little discussion that I'm at a loss as to what to do next.

My personal beef with the proposal is that I just can't relate to it.
So I don't see myself as being able to make any worthwhile contribution.

The proposal is to make the definition of "critical regression" depend
on the regtests and double the size of the regtests in order to catch
the same amount of regressions.  It does not change the principal
problem: if we now discover bad recently introduced behavior, we have
the choice not to create a regression test for it (in which case the
release can go forward), or the release will get blocked again once the
test has been created.

I think we can agree that refraining to create relevant regression tests
on purpose in order to make the release go forward would be
counterproductive.

There is a further proposal to reorganize the regtests, do more
programming work on them, and a "roadmap" focused on tieing things down
based on fuzzy goals like "GLISS" which is currently a random set of
proposals collected over the course of years.

Basically, this is a bunch of quite fuzzy new red tape only loosely
related to the tangle of red tape we have now holding up 2.16, and with
no view who would be motivated enough to implement the required
materials.

We are currently having a problem of people losing motivation, and so I
consider it not all too surprising that the level of excitement for
reviewing red tape is not particularly high, seeing how little
enthusiasm there is left for reviewing code.

> The whole point of GOP was to have a clear discussion with no
> surprises -- everybody would know when proposals would arise, there
> would be two weeks (or more!) of discussion, so that nobody would miss
> anything if they weren't paying attention to every little bug
> discussion email thread, etc.

Ok, here is my take on the current situation.  I'll start with a few
observations.

Releasing a stable release brings progress to LilyPond users.  LilyPond
users are the most promising clientele for recruiting future developers.
People start actively working with the versions they actually know and
use.  The less connections remain between the versions in the hand of
the users and the current development source, the less likely their own
work is suitable for eventual inclusion in LilyPond.  So we want to
avoid having stable versions that are quite outdated.

Regressions and bugs are a bad thing: we want to avoid them.

Detecting regressions and bugs is a good thing: we don't want to create
incentives to avoid detecting them.

What makes detecting bugs a good thing?  We gain the opportunity to fix
them, and we gain knowledge, the opportunity to evaluate their severity.

A stable release with severe bugs is a problem.  A stable release with
some bugs and regressions is pretty much unavoidable.

We don't differentiate the severity of regressions.  The smallest
regression discovered to occur under any circumstances will block a
stable release, which is a considerable cost.  That means that
discovering small regressions is doing the LilyPond community a
disfavor.

Policies should not turn good things into bad things.  So our knowledge
about regressions should be used for making a better qualified decision
when to release stable versions.  At the current point of time, it just
leads to the decision "no release".  A monkey could make such a
decision, and indeed, this has been a design goal of creating the
decision making procedures.  They work by following rules rather than
common sense.  There is no risk of making a bad decision, because there
is no risk of making a decision at all.  The facts make the decisions.

But facts don't have common sense.  They have no conscience.  They have
no sense of decorum.

When we design our rules for dealing with facts, we have to take this
into account.  And that requires leeway for incorporating common sense.
The rule sets you write are based on "let's assume that we don't have a
release manager at our disposal with any amount of clue and good sense".

I don't think that is a reasonable premise.  Not given the current
release managers.  And if we get others, they are not likely to follow
rules anyway.

Other large projects try to evade personal responsibility by making
timed releases.  And guess what: even though their rule sets are as
rigid as our bug-based releases, when a real whopper turns up, they
disband their rule set temporarily, and take a week or even a month
longer.  So their "facts don't affect the release date" is a white lie
overriden by common sense after all.  And I wish that we were able to do
a similar override to our "facts and rules are the only things affecting
the release date" stance when common sense makes this desirable.

If we work from the premise "users should be saved from regressions",
then we would backport any regression fix to a regression having occured
between 2.12 and 2.14 to the 2.14 stable branch.  We don't do that.  We
only fix the regressions in 2.16.  Which is of absolutely no advantage
to the user since the user does not even _have_ 2.16.

I don't see that the underlying problem is fixed by creating more rules
and policies.  So I am not overly motivated in getting involved in their
design, and I have the suspicion that I am not alone with that.

Let me quote from the News:

    @newsItem
    @subsubheading Release candidate 1 of 2.16 - LilyPond 2.15.8 released!  
@emph{Aug 01, 2011}

    LilyPond 2.15.8 is out; this is the first release candidate of
    the upcoming 2.16 stable release.  All users are invited to
    experiment with this version.  New features since 2.14.2 are
    listed in the @qq{Changes} manual on the website section about
    @ref{Development}.

    There are no known Critical issues with this release.  If no
    Critical bugs are found, then the official 2.16.0 release will be
    on 08 Aug 2011.  If you discover any problems, please send us
    @ref{Bug reports}.

    @newsEnd

What is the meaning of "release candidate"?  It is soon a full year
since this "release candidate" was announced, and the current state of
LilyPond is far removed from the state at that time.

We are obviously doing something wrong, and it is costing us users,
developers, and motivation.  And I don't see that fine-tuning the
respective GOP proposals will bring them back.

At the current point of time, I don't see that we can win much by
streamlining batch processes for turning out decisions.  They have not
arrived at a state where their output is convincing, so maybe it is time
to arrive at a few decisions manually.

I don't see how we can otherwise get back to a state where working on
bugs and regressions and other parts of LilyPond is not a punishment
and/or guilt trip, but actually rewarding.

-- 
David Kastrup