Re: [Gnu-arch-users] GCC v. Arch address@hidden: Regressions on mainline]

[Andrew Suffield]

On Wed, Jun 23, 2004 at 11:49:23AM -0500, Charles Duffy wrote:
> On Wed, 2004-06-23 at 11:15, Andrew Suffield wrote:
> > On Wed, Jun 23, 2004 at 12:10:28PM -0400, Colin Walters wrote:
> > > You could also imagine a separate category for the test suite, which
> > > developers could also commit to.
> > 
> > ...yes, that emphasises the problem.
> > 
> > The *result* of 'make check' is not static at all. Consider the case
> > where a new test has just been added, which (a) fails, and (b) is a
> > regression from the previous release.
> 
> Simple solution, though it uses some upcoming functionality:
> 
> Require the change to the test suite, if it fails with the head of the
> development line, to either (1) have a flag that says "permit-failures",
> or (2) be checked in atomically [this is where the upcoming-features bit
> comes in] with a fix to the head of the development line.

(1) is not something that you want to fold into the test suite. This
is a property of a (release) branch. It is *not* a property of the
test suite itself. The XPASS/XFAIL mechanism gcc uses is essentially
wrong, and a reflection of their currently broken tools. A failure is
an expected failure on a certain set of branches only; managed
development on parallel branches requires this.

On general principle it's kinda-sorta right, but actually doing it
right depends on a pile of infrastructure that doesn't exist yet.

(2) is impractical, because it assumes all bugs are filed with
patches. You want the test in there as quickly as possible, so that
the system can start working and provide you with the information you
need to write the patch (eg, the changeset which broke it).

> I'm quite (over?)confident that this is a solvable problem, as long as
> the corner cases get addressed.

Oh, it's solvable, but not within the existing infrastructure, and
'make check' does something entirely different. That's for users and
developers.

-- 
  .''`.  ** Debian GNU/Linux ** | Andrew Suffield
 : :' :  http://www.debian.org/ |
 `. `'                          |
   `-             -><-          |

signature.asc
Description: Digital signature