Re: Problems with make test

[Phil Holmes]

----- Original Message ----- 
From: "Graham Percival" <address@hidden>
To: "Phil Holmes" <address@hidden>
Cc: "Devel" <address@hidden>
Sent: Tuesday, July 31, 2012 4:48 PM
Subject: Re: Problems with make test


> On Tue, Jul 31, 2012 at 11:22:25AM +0100, Phil Holmes wrote:
> > So I open collated-files.html, to find that there are no images to
> > eyeball. The HTML expects PNG files, but very few are created (in
> > particular, the one I want...).
>
> collated-files.html only shows the differences, not the actual
> files.  Furthermore, these differences are calculated based on the
> .eps or something like that, and text is *not* included in the
> comparison!  For various reasons, the original authors of this
> system deliberately did not use a pixel-based checker.

After make test, out-test/collated-files.html currently shows a complete 
list of all the compiled regtests, but with broken links to the tests 
themselves and to the images.  make test doesn't do comparisons, AFAIK.  My 
recent experience with the failing make test tells me that debugging 
problems with make test is now _much_ harder than with make doc, and it 
seems to me that being unable to run make test and view the output is not 
helpful.  I've got a patch ready that fixes this, by adding png to the 
output formats and jiggling other options.  Because of the additional 
output, it takes a bit longer (16% = 17 seconds on my machine).  If I delete 
the option of the PS output, it actually runs just under 10% quicker.  I'll 
put this for review unless you say it's a waste of time.

--
Phil Holmes