Re: 2.16 release criteria

[Graham Percival]

On Wed, Mar 21, 2012 at 01:06:42AM +0100, David Kastrup wrote:
> 
> I have the fear that the desire to get to this state might prompt some
> regression fixes that have not necessarily gotten all the diligence that
> would have been desirable.

This is a valid fear in general, but I haven't seen it happen yet.
Granted, I don't usually review scheme or C++ patches, so perhaps
people have been sneaking bad bugfixes in that way?  But I
somewhat doubt that.

> So I am not sure that a "timed release"

No.  Absolutely not.

Yes, it might be good to change the release policy.  But I will
not accept any discussion along those lines.  We discussed matters
to death in GOP.  It hasn't even been 12 months!  What's the point
of having a serious policy discussion if it's going to change in a
few months?

In the summer, I will begin GOP2, and we will begin by reviewing
every single policy decision made in GOP.  It will be understood
that whatever policies we agree upon in GOP 2 will hold for at
least the next year.  We may end up having a yearly review of such
policies.

> On the plus side, regressions are being addressed vigorously right now.
> Other bugs, however, get to see this vigor as well, leading to more
> regressions in their wake.

I think we're looking at about 30% Critical regressions due to
code in the past year.  Solution?  More eyes on reviews and/or
more regtests.

The bulk of Critical regressions happened during the long 2.13
process.  Those block a stable release on the basis of last year's
policy discussions.  To make matters worse, we've begun a big
review of the regression tests.  I guessimate that we currently
have between 5 and 20 broken regtests; the regtest review will
probably find those.

In the long term, I think we're doing fine.  For the first time
ever, we're not regularly breaking regtests.  I cannot emphasize
how important this is -- back when I was handling bugs, I would
see 1-2 broken regtests every devel release.  Unforuntately we're
in for some more pain in the next few months as we discover
previously-broken tests, but once that's shaken down, we'll have a
trustworthy set of regression tests.

- Graham