Hey Lilypond devs!
GSOC student here. I have some questions regarding musicxml export, since
David is out of town without internet. Here is his outline for the first
half of the project (at the bottom) and my subsequent reply + questions (at
the top). In particular, I just want to make sure I fully understand what
needs to be done with a simple example. Thanks for any help!
-David
---------- Forwarded message ----------
From: David Garfinkle <address@hidden>
Date: Wed, Jun 3, 2015 at 7:34 PM
Subject: Re: Ekiga chat
To: David Kastrup <address@hidden>
Hi David,
I think I get what's going on and what I need to do. But I would like to
make sure with an example.
Given this input file "test1.ly"
\version "2.18.2"
\score{
\relative{a4}
}
1. a file similar to dumpit.ly would find&separate the {} blocks? and for
our simplest case we're just looking at a score block with one note
2. then for this score block we have a scheme representation that looks
like this:
(make-music
'SequentialMusic
'elements
(list (make-music
'NoteEvent
'pitch (ly:make-pitch -1 5 0)
'duration (ly:make-duration 2 0 1))))
which further defines a C++ representation, that I don't need to work
with..?
3. And this is where I want to make really sure I know what's going on. We
want to turn the scheme function representation, using the display-lily
machinery, into an SXML representation which looks like this: (i think?)
(list (make-music
(list 'SequentialMusic
(list 'elements
(list (make-music
'NoteEvent
(list 'pitch (ly:make-pitch -1 5 0)
(list 'duration (ly:make-duration 2 0 1)))))
or equivalently,
'(make-music
(SequentialMusic
(elements
(make-music
NoteEvent
(pitch (ly:make-pitch -1 5 0))
(duration (ly:make-duration 2 0 1))))))
... so that each scheme function is now the first element in a series of
nested lists
and this will be done in a guile-1/lilypond script
4. then a guile-2 script uses (sxml simple) module and calls sxml->xml, and
with the above example, turns our music into
<make-music>
<SequentialMusic>
<elements>
<make-music>
NoteEvent
<pitch>
<ly:make-pitch>-150</ly:make-pitch></pitch>
<duration>
<ly:make-duration>201</ly:make-duration></duration>
</make-music>
</elements>
</SequentialMusic>
</make-music>
And then of course before we make it xml, we need to do an sxml->sxml
transformation which maps lilypond to the musicXML specifications.
Does this look at all right, or is there a big gap in my understanding? I
am rather confused as to what (make-music) is doing in the xml, and what it
would mean in musicXML, so I feel like I have missed something.
Thanks for any help!
Best,
David
On Tue, Jun 2, 2015 at 4:00 PM, David Kastrup <address@hidden> wrote:
Ok, sketching what I consider useful next.
If you run
lilypond -d include-settings=dumpit.ly xxx.ly yyy.ly
with some LilyPond input files xxx.ly and/or yyy.ly and the following
file "dumpit.ly"
then you'll see that each contained score will be output to the
terminal, preceded by a statement which file this is intended for. Now
instead of announcing writing to a file one will actually write to a
file, and instead of using display-lily-music one will use something
else. Also one will not just extract the music from the score but also
various headers, variables and other stuff.
But the music in the score is the start.
The tasks we have is to convert music data structures to XML. And we
may have different forms of XML, and they may not correspond
structurally with the music we start with so we might need to recognize
patterns/structures and convert to other patterns.
Now in the long term we want to convert also from XML back to LilyPond.
And for producing syntactically valid and nicely formatted XML, it makes
sense to use a preexisting library producing XML output from Scheme data
structures.
music -> scheme representation of XML -> XML
There are currently two XML libraries I see: namely SLIB and SXML, and
the latter is only available for GUILE-2 if I am not mistaken. The
representation of SLIB and SXML is quite similar: changing from SLIB to
SXML should not be much of a problem once LilyPond has been switched to
GUILE-2. The time frame for that is dubious.
Now the GUILE-1 SLIB however is not useful for our task since it appears
limited to reading and interpreting XML. It produces an SXML rendition
of an XML file but is not capable of writing one.
That is, uh, unfortunate. Darn.
The GUILE-2 SXML library has a pattern matching and transform library
that has a few perks (namely that stuff like attribute order do not
matter when matching XML) but those are not really important for
converting from music->XML since om that direction the order of any
attributes will be the same every time since it is under our control,
and MusicXML does not use attributes much anyway I think.
At any rate, the basic music->scheme/XML transformation should be doable
by copying and changing the display-lily-music machinery in
scm/define-music-display-methods.scm and scm/display-lily.scm (currently
producing strings from Music) to a generate-sxml machinery (producing
lists from Music). In a first approach, it would seem reasonable enough
to use the same programmatic structure and just replace its respective
outputs, formatted strings, by the respective list representation in
SXML.
So that should give us a basic path music->SXML which then may need
another transformation layer at the SXML level in order to get the
structure of MusicXML. This structural transformation could be done at
other layers in the conversion, but since a future XML input option to
LilyPond will require the reverse direction, it makes sense to have this
transformation not combined with a conversion layer since then the
patterns for recognizing the structures would be for different formats
when converting music->XML and XML->music. And for maintenance reason
we'd want both of those to be using the same kind of transform
mechanism.
So since we will ultimately do the "structural transformation" as a
stage from SXML to SXML, we do not need to design it into the first
prototype producing XML.
What's really awful is that right now we cannot use GUILE-2 in LilyPond,
and I was mistaken about the SXML capabilities of GUILE-1. So we'll
need to produce the SXML representation (which is just a list) in
LilyPond/GUILE-1, write it out to a file or a pipe (using "write" or
similar functions writing out Scheme data), read it back into a GUILE-2
script that does the further processing/transformation and write out the
resulting XML.
Ugh. That's ugly. But I can't promise a time frame for a working
GUILE-2 inside of LilyPond: that's an urgent project but it has been
urgent for years.
So we'll need to split this into a GUILE-1 and a GUILE-2 part until the
GUILE-2 migration of LilyPond completes independently, or we need to
skip the intermediate SXML stage/usage altogether and produce XML
strings right away. But I think that that approach would prove to be a
bad choice for longterm maintenance, so I'd rather go with split parts.
So it probably makes sense to first write a GUILE-2 standalone
application first that reads in a file in SXML (which should just be one
"read" statement reading in a Scheme list) and uses the SXML writer
(described in the GUILE-2 manual in the library section) for producing a
corresponding XML file.
Then write a function in GUILE-1 that writes out a Scheme list and calls
the GUILE-2 script. If we have that, we basically have a function in
GUILE-1 that can be called with SXML and write it out to XML (by writing
out the SXML to a temporary file or a pipe and calling the GUILE-2
script on it).
Ok, try digesting that, see whether you get the points of the, uh,
somewhat changed plan and whether you see yourself approaching it in
that manner or whether you need more pointers or think doing it
differently may make sense.
I sure wish my original plans would have corresponded better with
reality, but then this is also supposed to be relevant experience for a
later job and, uh, "drop the initial plan and come up with something
that has a chance of working" is quite a standard procedure.
All the best
--
David Kastrup
_______________________________________________
lilypond-devel mailing list
address@hidden
https://lists.gnu.org/mailman/listinfo/lilypond-devel