Re: clear policy discussions

[Graham Percival]

On Tue, Jul 10, 2012 at 09:06:10PM +0200, David Kastrup wrote:
> Graham Percival <address@hidden> writes:
> 
> > My personal opinion is that we should not discuss major policy
> > changes in the middle of a thread about dots and staff sizes.
> 
> It was in the course of changing the status to "Critical".

That was not obvious to a casual reader.  If somebody only checked
the lilypond mailing lists once a week, they'd have over 100
emails to skim through, and I would delete the email thread called
"issue 2648 repeat dots and staff sizes" without glancing at any
message bodies.

> > There is a proposal to change the stable release policy:
> > http://lists.gnu.org/archive/html/lilypond-devel/2012-06/msg00462.html
> > http://lilypond.org/~graham/gop/gop_3.html
> >
> > That proposal is currently "under debate", although there was such
> > little discussion that I'm at a loss as to what to do next.
> 
> My personal beef with the proposal is that I just can't relate to it.
> So I don't see myself as being able to make any worthwhile contribution.

That itself is a valuable contribution!  If a proposal is so
vague, or poorly worded, or just plain confusing that a main
developer like you "can't relate to it", then there's no way that
we should be adopting such a proposal.  At the very least the
proposal should be rewritten; at most it should be scrapped
outright.

> The proposal is to make the definition of "critical regression" depend
> on the regtests and double the size of the regtests in order to catch
> the same amount of regressions.  It does not change the principal
> problem: if we now discover bad recently introduced behavior, we have
> the choice not to create a regression test for it (in which case the
> release can go forward), or the release will get blocked again once the
> test has been created.
>
> I think we can agree that refraining to create relevant regression tests
> on purpose in order to make the release go forward would be
> counterproductive.

The proposal does not agree with you on that point.  The proposal
is saying "let's not care about bad behaviour.  Let's only care
about the functionality that is currently tested.  If there's some
bad behaviour that's not covered by an existing regtest, we'll
ignore that behaviour for the purpose of making a release".

In other words, the proposal is suggesting that the cost of not
making stable releases far outweighs the cost of shipping bad
behaviour.

You evidently do not agree with that judgement, which is totally
fair.  Let's talk about this more.


** NB: I've tried to write about the proposal in the third-person,
and I would ask you to do the same.  Please do not write "you're
insane if you think it's ok to ship broken code just because it's
not in a regtest"; instead, please write "the proposal is insane
if it results in shipping broken code just because it's not in a
regtest".


> There is a further proposal to reorganize the regtests, do more
> programming work on them, and a "roadmap" focused on tieing things down
> based on fuzzy goals like "GLISS" which is currently a random set of
> proposals collected over the course of years.
> 
> Basically, this is a bunch of quite fuzzy new red tape only loosely
> related to the tangle of red tape we have now holding up 2.16, and with
> no view who would be motivated enough to implement the required
> materials.

That is true, and it is an excellent critique of the proposal.
Clearly it cannot be adopted in its present form, so this is a big
step forward relative to where we were yesterday.


> Releasing a stable release brings progress to LilyPond users.  LilyPond
> users are the most promising clientele for recruiting future developers.
> People start actively working with the versions they actually know and
> use.  The less connections remain between the versions in the hand of
> the users and the current development source, the less likely their own
> work is suitable for eventual inclusion in LilyPond.  So we want to
> avoid having stable versions that are quite outdated.
> 
> Regressions and bugs are a bad thing: we want to avoid them.
> 
> Detecting regressions and bugs is a good thing: we don't want to create
> incentives to avoid detecting them.
> 
> What makes detecting bugs a good thing?  We gain the opportunity to fix
> them, and we gain knowledge, the opportunity to evaluate their severity.

Agreed stongly!  If you don't mind, I think I may copy&paste this
verbatim into the "motivation" section of whatever revised
proposal comes next.

> A stable release with severe bugs is a problem.  A stable release with
> some bugs and regressions is pretty much unavoidable.

I agree with qualifications: a stable release with severe bugs is
a problem, but this problem may or may not be very large.  I mean,
it may be desirable to "bite the bullet" by accepting that it is a
problem but that the alternative(s) are worse.

> We don't differentiate the severity of regressions.  The smallest
> regression discovered to occur under any circumstances will block a
> stable release, which is a considerable cost.

Agreed.  It seems desirable to avoid this cost, which requires
some change of policy.

> That means that discovering small regressions is doing the
> LilyPond community a disfavor.

Only under the current policy, though.

> Policies should not turn good things into bad things.  So our knowledge
> about regressions should be used for making a better qualified decision
> when to release stable versions.  At the current point of time, it just
> leads to the decision "no release".  A monkey could make such a
> decision, and indeed, this has been a design goal of creating the
> decision making procedures.

That is absolutely true.  The goal was to have a crystal-clear
process, and monkies are noted for their ability to observe solid
materials whose molecules are arranged in an orderly, repeating
pattern.

> When we design our rules for dealing with facts, we have to take this
> into account.  And that requires leeway for incorporating common sense.
> The rule sets you write are based on "let's assume that we don't have a
> release manager at our disposal with any amount of clue and good sense".

"... and who doesn't know about lilypond internals + code, or
documentation, or translations, or build process, etc".

> I don't think that is a reasonable premise.  Not given the current
> release managers.

I disagree here.  There's a reason that I never review scheme
patches, and only occasionally look at C++ patches where my only
possible contribution is "hey, you didn't initialize this
variable".

I *don't* have any amount of clue when it comes to lilypond
internals.  Is it safe to backport your latest fix to the parser?
I have no clue; maybe that requires some other scheme internals
change.  Maybe it doesn't.  I don't have the background to tell,
nor am I sufficiently motivated to learn that background at the
moment.

For better or worse, the current release manager *is* a trained
monkey.  What about the next one?  Phil has been learning how to
do it, and I certainly won't call *him* a trained monkey... but
his expertise is documentation and build systems.


That said, there is no reason why "the person who does a
copy&paste from CG 11.2" has to be "the person who decides what
gets backported" or even "the person who decides what version
number to slap an a release".  It's beginning to look like we
really *should* split up those roles.

> Other large projects try to evade personal responsibility by making
> timed releases.  And guess what: even though their rule sets are as
> rigid as our bug-based releases, when a real whopper turns up, they
> disband their rule set temporarily, and take a week or even a month
> longer.

As an unimportant aside: really?  I can't recall Ubuntu or OpenBSD
ever postponing a release.


> If we work from the premise "users should be saved from regressions",
> then we would backport any regression fix to a regression having occured
> between 2.12 and 2.14 to the 2.14 stable branch.  We don't do that.  We
> only fix the regressions in 2.16.  Which is of absolutely no advantage
> to the user since the user does not even _have_ 2.16.

Agreed that this is a problem.  At the moment we don't really have
any "stable version".

> I don't see that the underlying problem is fixed by creating more rules
> and policies.  So I am not overly motivated in getting involved in their
> design, and I have the suspicion that I am not alone with that.

Well, removing policies is also a policy.  Perhaps the best thing
to do would be to adopt this policy for stable releases:

    Person X will decide when to make a stable release.
(the end)

I'm certainly not opposed to such a policy!  I would want that
policy to be clearly stated, discussed for any downsides (do we
trust person X to be competent?  to have enough time?  maybe we
should appoint person X for 6-month intervals?  or do it on a
per-branch basis?  etc.).

If the final answer was "David Kastrup will have sole
consideration for the 2.16.x releases (including when to make the
release, what patches go into it (i.e. backports), etc); Phil
Holmes will have sole consideration for the 2.17.x release; at the
moment we will reserve judgement for any 2.18 or 3.0 releases",
then I wouldn't object.


> We are obviously doing something wrong, and it is costing us users,
> developers, and motivation.  And I don't see that fine-tuning the
> respective GOP proposals will bring them back.

I agree that we're doing something wrong.  All I want to make sure
that we have a clear and open discussion about how to fix it, so
that no nobody (particularly "casual developers" who don't read
the mailing list all the time) is unpleasantly surprised.

I evidently wasn't clear in introducing GOP and their policies; a
complete rejection of any proposal is certainly valid!  The GOP
proposals are intended to begin debate, not end them.

- Graham