Regtest rating project

[Phil Holmes]

We're getting to the time where the Grand Regression Test Checking (TM) 
project 
(http://lilypond.org/doc/v2.15/Documentation/contributor/grand-regression-test-checking 
) has got enough results to allow us to use it to fix poor regtests.  We 
therefore need to agree how to get agreement on the changes needed to the 
regtests.  To put it into perspective, there are over 1000 regtests, of 
which 200 have been shown to need attention.  My thoughts on our options:

Custom web application
================
I write an application that displays regtest images, texidoc, comments and 
suggested change, and interested parties can add comments
Pros
----
Should be possible to review a lot of proposed changes reasonably quickly, 
and get agreement
Cons
----
I need to write another application.  Authentication likely to be a pain.

Custom app with Google authentication
==========================
Pros
----
As above, plus gets rid of authentication issue

Cons
-----
I need to learn Google auth _then_ write another application

Wiki
===
Pros
----
Should be possible/easy to set up

Cons
----
Easy to lose track.  Authentication.

Use the Issue Tracker
==============
Pros
----
Feasible.  Would keep everyone informed of proposed changes and allow 
discussion

Cons
-----
If we use an issue per regtest, we're likely to overwhelm the team with 
emails and discussion (in particular those with no specific interest in this 
work) and also the Bug Squad with verifying.  If we bulk up regtests into an 
issue, I think it would get confusing.  We could also use up all the 
remaining attachment space instantly.

New Google Project
==============
This occurred to me only recently.  What about setting up a new Google 
Project called "LilyPond Regression Test Improvement".  This would require a 
new mailing list to avoid the problems above.  We could then add an issue 
per regtest, and only people who want to subscribe would need to subscribe 
to the list.

I think the final suggestion is my favoured one.

--
Phil Holmes