I've been looking at how the regression test comparison works. The
first thing I find is that we have 2 effectively duplicate, but
different, pages on running regtest comparisons:
http://lilypond.org/doc/v2.15/Documentation/contributor/verify-regression-tests
http://lilypond.org/doc/v2.15/Documentation/contributor/regtest-comparison
I think the latter is probably more accurate. I think it would be best
to delete one and point to the other?
I've also been benchmarking. For example, I know that make CPU_COUNT=9
test is _much_ faster than make test, but the make -j9 test isn't worth
doing - most of the time is spent building the single regtest document,
which lilypond parallelises much better than make. I have had errors
using -jX so am slightly suspicious of that option. I would like to add
the best way to parallelise the test process to the CG.
I've also been looking at how output-distance works. Does anyone now
understand what this actually does? From following the code, it looks
to me like it doesn't actually compare images - it compares the
.signature files, and if there's a difference over the threshold, it
creates an image of the original and changed file, and then makes a
"ghost" version of the change to overlay on the original. Does this
seem correct. Worth documenting?
--
Phil Holmes