Re: Revised version of waveform renderer on Rietveld that uses glpk

[Mike Solomon]

Responses inlined:

On 7/2/10 4:54 PM, "David Kastrup" <address@hidden> wrote:

> Mike Solomon <address@hidden> writes:
> 
>> http://codereview.appspot.com/1720046
>> 
>> Hey developers,
>>     Today I had time to finish version 2 of my waveform patch that uses glpk
>> to do smooth linear interpolation of waveforms.  As there is a dependency
>> freeze & glpk is ultimately too big & too everchanging to be part of
>> lilypond, this won't make it into 2.14, but I'd like to get it into a
>> development version ASAP.  I've already composed a piece w/ the old version
>> of the patch & the new version has rendered said piece even prettier.
> 
> What musical function serves this patch/notation?

In contemporary music, I've worked with several composers that use
representations of waveforms in their scores so that players can follow
along.  Only really doable in Finale or Sibelius, the creation of said
representations is a tedious process that requires exporting an image of a
waveform and stretching it to fit with the music.  If the systems change, a
new batch of images need to be made and stretched.  As far as musicology
goes, I know of at least one project going on in France that allow for the
mixing of scores and audio representations:

The dynamic score (http://www.grame.fr/Recherche/Programmes/)

There are a few other prjects that I remember seeing at Journées
Informatique Musicale in Rennes this year that were also along this line.
It is definitely in the air (at least in certain communities).

I believe that this type of need is widespread.  And, as the algorithm in
waveforms is written to work for anything that is created from a lot of
small samples, it can be used with RGB images and SVG/PS filters as well to
create graphical notation.  I will be creating examples of this in the not
too distant future.

> There is no documentation as far as I can see, and I have the strong
> suspicion that this is exclusively an ornamentation.

I'm sorry for the lack of documentation: the patch is hot off the press.  If
I had to sum it up in a phrase, this patch allows for the smooth stretching
and relaxing of sampled images in the context of a vector-graphics based
program.
Trills are exclusively ornamentation, and I <3 trills :-)

> It seems nice to be able to add this sort of thing to Lilypond, but I
> think it rather strongly demonstrates Lilypond's lack of modularity:
> this sort of thing should sit in a separate directory and be loaded
> on-demand under user control without needing any resident code parts
> when people don't use it.

I completely agree that modularity rocks, but to be fair, I think that
lilypond is excellently modular through Scheme.  In fact, if scheme had a
linear programming library, I could have created 100% of this in scheme
using various overrides and hijackings of other spanners, although it would
have run considerably slower for certain problematic waveforms that need
many rounds of smoothing.

> It makes no sense to maintain such a subfeature in a separate _branch_,
> this should be a sub_project_ (in git terminology).  Including or not
> including such a feature in Lilypond should not be a question of merging
> some branch (with all the conflict possibilities), but rather of
> including some directory in the hierarchy.  The difference at run time,
> again, should be the presence or absence of files, preferably in a
> separate directory of the Lilypond hierarchy.

This is an interesting route to explore - it raises the question: what is
modular to lilypond and what is central?  I believe there are many elements
of the code (ie the Vaticana stuff) that could be treated this way, but they
don't bother me in lily/ simply because I think that their presence makes
lilypond more robust w/o slowing it down appreciably.

and from Graham's e-mail...

> I haven't looked at the patch, but I'm guessing that the
> non-modularized parts is the interaction with glpk.  Is there any
> way that scheme could call a natively-install glpk library (i.e.
> not the -devel version) ?  That might make this feature
> modularizable, and incidently might even allow it to get into
> 2.14.

Your evaluation is right on the money.  I can absolutely make this happen
with a call to glpsol from scheme (or something of this ilk).  But the
danger there is that I have no clue how that works on non Linux / OS X
machines.  And, there are a couple other technical advantages to using C++:
most notably the multimap that exists in the waveform class which allows a
waveform part to see the calculations of its whole.  Furthermore, given the
way lilypond is designed, I think that it makes sense to write engravers in
C++.  Perhaps I could change the name, however, to "sampled-object-engraver"
or something of the ilk to allude to its broader applicability.

The reason I use glpk is because the sampling rate must remain above 0 at
all times.  The previous version did not use glpk, and met this constraint
in a rather unequal way by lifting the sampling rate above 0 locally
wherever it dipped below, which caused jolts in the look of certain
waveforms.  The new version is much smoother, and would be even smoother if
I could use CVXOPT from python to do semi-definite cone programming, but
calling the Python interpreter from C to run CVXOPT seems kinda bogus & I am
not good enough at coding to understand (in the time I have to devote to
lilypond) the other convex optimization libraries that exist here and there
online.

In general, anytime there is a concern that something may dip below zero in
lilypond (distance between two objects comes to mind), one way to solve said
class is to create a semidefinite or quadratic program with the constraint
that the distance between said things be greater than 0.  Such type of
thinking could be a very neat and powerful way to facilitate this aspect of
lilypond's layout process.

Parting shot: waveform allows for the typesetting of sampled images with
smooth linear interpolation based on marked points in a score.  This can be
used for any graphical image generated from discrete samples.

Thank you for your consideration and feedback!
~Mike