Re: lilypond-book is hosed

[Graham Percival]

On Thu, Dec 24, 2009 at 01:06:19AM +0100, John Mandereau wrote:
> Le mercredi 23 décembre 2009 à 23:17 +0000, Graham Percival a écrit :
> > Yes; that's why we changed the hash calculation from self.ly() to
> > self.full_ly()
> 
> You meant the opposite.

err, yeah.

> >   -- produced lily-abcd1234 filename would not
> > depend on the fragment options,
> 
> This is wrong.  lilypond-book relies on hashing to decide whether a
> snippet should be (re)processed by lilypond, so framgent options that
> change processing by lilypond must be taken into account in the hash;

Ah, we're talking about two different kinds of hashes.  One hash
decides if the snippet should be (re)processed -- I don't care
about that one.  Do whatever seems sensible for that.

The other kind of hashing produces the lily-xxxxxxxx.ly filename
based on the contents.  *That's* the kind that we (intentionally)
changed to be independent of snippet options, in order to have the
same filename produced for regtests with different options.

That change seems to have disturbed the first kind of hashing,
though.


** edit: rather, there are two *goals* of hashing in
lilypond-book.  Technically, they seem to be generated via the
same mechanism, which is the cause of this whole problem.  But we
avoid that problem by using the regtest filenames, so that's fine.

> > I mean not using xx/lily-xxxxxxx.ly files for the regtests.
> > Instead, use files of the form:
> >   out/lybook-db/regtest/xxxxxxxxxxxx.ly
> > where the filename is used directly.  Like:
> >   out/lybook-db/regtest/ancient-accidental.ly
> >   out/lybook-db/regtest/slur-broken.ly
> >   out/lybook-db/regtest/tuplet-numbers-avoid-slurs.ly
> 
> This is doable and interesting to do for writing files in
> input/regression/out-www, but this makes no sense in out/lybook-db,
> where files should have a name predictable by lilypond-book (namely
> obtained using a hash), so it can decide for each snippet whether it's
> already been compiled or not.

Doesn't input/regression/out-www/ use out/lybook-db ?  I thought
it did, but I'm happy to admit my ignorance about this.

The bottom line is this: I'd like a regtest processed with one set
of options to have the same filename as that same regtest
processed with another set of options.  I don't care what you do
to either set of hashes, as long as that goal is met.  :)

> > ?  All that will happen is that if somebody modifies a regtest
> > (like adding an additional bar to slur-broken.ly), that regtest
> > will show up in the regtest comparison for the next version.  On
> > average I think that adds 3-4 images per release?
> 
> You misunderstood me.  I was afraid that the regtest comparison script
> would issue false positive differences because of a change in
> lilypond-book hashing, but maybe this fear is purely virtual.

Whatever happens, I'll need to generate a new set of 2.13.9
regtests with the new filename-hashing (or
regtest-filename-keeping) in order to compare them with the
2.13.10 regtests.  The actual comparison is done by imagemagick
comparing the image files, so the first kind of lilypond-book
hashing shouldn't matter.


> > The regtest naming issue should be fairly simple: add a new option
> > to lilypond-book that turns off the snippet hash calculation.
> > Wherever hash(self.ly()) is called, check the option and return
> > the filename if it's set.
> 
> No, as I argumented at the beginning of this message, this would
> completely break the idea of a shared snippet database.  However, a
> lilypond-book command-line option to write nice filenames in
> input/regression/out-www is fine,

Sorry, I was only intending to propose this for the regtests, not
the normal docs.

> and will achieve what you explained
> above (regtest comparison is based on input/regression/out-www or its
> mirror in out-www/offline-root, right?).

I think it's actually via "make test", following by tarring the
produced input/regression/out-test.  I'm not totally certain about
this, though.

Cheers,
- Graham