MIDI output: min/max MIDI volume levels, dynamics of polyphonic passages within a single Voice

[g034737]

Hi,

First of all, thanks to all the developers and the community for
making and maintaining this absolutely wonderful tool!  I have never
used any music typesetting program with which I could convert
vocal scores into MIDIs (which I use for practicing) with such ease
and speed, thanks to LilyPond's text-based input language.

Having also tried the MIDI conversion on parts of some orchestral scores
(and for learning how to express various things in layout), I have,
however, observed strange behavior related to controlling MIDI volume
levels for instrument equalization (as documented in the Notation
Reference).  More precisely, it feels as if the Dynamic_performer
sometimes fails to respect custom volume ranges defined using  
Staff.midiMinimumVolume and Staff.midiMaximumVolume (or by using a  
custom
Score.instrumentEqualizer).  That is, the Dynamic_performer appears to
let instrument volume levels exceed (or drop below) the defined ranges --
at least in connection with (de)crescendi that are not terminated with
an absolute dynamic mark (but \! instead).

Since existing scores often contain e.g. hairpins that do neither begin
nor end in an absolute dynamic mark, trying to replicate such notation
faithfully in LilyPond may lead to surprising end results in MIDI as,
for example, a sequence of (de)crescendi without any absolute dynamic
marks can cause the relative volumes of different instruments to easily
drift "off balance" (and often remain in such a state until the volume
of an instrument then suddenly changes back to a "properly equalized"
level at the next absolute dynamic mark).

An example:

------

\version "2.12.3"

% define a custom equalizer as instructed in the Notation Manual

#(define my-instrument-equalizer-alist '())

#(set! my-instrument-equalizer-alist
 (append '(
   ("flute" . (0.5 . 0.6))
   ("harpsichord" . (0.2 . 0.6)))
  my-instrument-equalizer-alist))

#(define (my-instrument-equalizer s)
  (let ((entry (assoc s my-instrument-equalizer-alist)))
   (if entry
    (cdr entry))))

\book {
  \score {
    <<
      \new Staff {
        \set Score.instrumentEqualizer = #my-instrument-equalizer
        \set Staff.midiInstrument = #"harpsichord"
        \new Voice {
          % (1)
          <c' e' g'>4_\ppppp
          % the alternation between chord notation and temporary
          % polyphony within a voice will be explained below
          << { c' } \\ { e' } \\ { g' } >>
          <c' e' g'>
          << { c' } \\ { e' } \\ { g' } >>

          s2

          % (2)
          % both instruments at ffff: it would seem reasonable to expect
          % that the volume of this staff's instrument cannot rise (much)
          % above this level
          <c' e' g'>4_\ffff
          << { c' } \\ { e' } \\ { g' } >>
          <c' e' g'>
          << { c' } \\ { e' } \\ { g' } >>

          s2

          % (3)
          % however:
          <c' e' g'>4_\<
          << { c' } \\ { e' } \\ { g' } >>
          <c' e' g'>4\!_\<
          << { c' } \\ { e' } \\ { g' } >>

          s2\!

          % (4)
          % further crescendo...
          <c' e' g'>4_\<
          << { c' } \\ { e' } \\ { g' } >>
          <c' e' g'>\!_\<
          << { c' } \\ { e' } \\ { g' } >>

          s2\!

          % (5)
          % return to ffff (just to demonstrate the difference between
          % the start and end volume)
          <c' e' g'>4_\ffff
          << { c' } \\ { e' } \\ { g' } >>
          <c' e' g'>
          << { c' } \\ { e' } \\ { g' } >>
        }
      }
      \new Staff {
        \set Staff.midiInstrument = #"flute"
        \new Voice {
          c''1_\ffff
          s2
          c''1
          s2
          c''1
          s2
          c''1
          s2
          c''1
        }
      }
    >>
    \midi { }
  }
}

------

The example consists of five stages to demonstrate the changes in
volume:

- Stage (1) is just to demonstrate the difference between the volume
  levels of the solo instrument (the flute) and the accompanying
  instrument (the harpsichord).

- At stage (2), the volume of both instruments is set to \ffff.

- At stages (3) and (4), the volume of the accompanying instrument is
  increased using multiple consecutive crescendi terminated using \!.
  Here, I would expect that the equalization of instruments would help
  to prevent the volume of the accompanying instrument from being
  increased "too much"...

- At stage (5) we simply return to the volume levels set at stage (2).


The MIDI produced from this example with LilyPond 2.12.3:

http://koti.welho.com/htauriai/test-2.12.3.midi

At stage (2), the solo instrument is reasonably audible (at least on
my MIDI configuration) even though both instruments are played at
\ffff.  However, in stages (3) and (4), the volume level of the
accompanying instrument only seems to continue to increase in spite
of the attempt to apply equalization to the instruments.

I had a look at the source code to see whether I could understand what
could be happening here (I must confess that I had absolutely no
previous knowledge on the LilyPond internals).  From the source code,
I eventually concluded that the Audio_span_dynamic::render function might
indeed not implement the kind of volume level restrictions that I had
expected, when the function is applied to dynamic span events that do not
end in an absolute dynamic event.  To experiment with this, I tried  
the following change (which simply extends each Audio_span_dynamic with
explicit minimum and maximum volumes, which are initialized using the  
equalize_volume function in Dynamic_performer):

------

diff -Nru lilypond-2.14.2//lily/audio-item.cc tmp//lily/audio-item.cc
--- lilypond-2.14.2//lily/audio-item.cc 2011-07-26 01:42:15.000000000 +0300
+++ tmp//lily/audio-item.cc     2011-09-11 17:21:17.000000000 +0300
@@ -79,9 +79,11 @@
 {
 }

-Audio_span_dynamic::Audio_span_dynamic ()
+Audio_span_dynamic::Audio_span_dynamic (Real min_volume, Real max_volume)
 {
   grow_dir_ = CENTER;
+  min_volume_ = min_volume;
+  max_volume_ = max_volume;
 }

 void
@@ -135,7 +137,7 @@

   Real start_v = dynamics_[0]->volume_;
   if (dynamics_.back ()->volume_ < 0)
-    dynamics_.back ()->volume_ = max (min (start_v + grow_dir_ *  
0.25, 1.0), 0.1);
+    dynamics_.back ()->volume_ = max (min (start_v + grow_dir_ *  
0.25, max_volume_), min_volume_);

   delta_v = dynamics_.back ()->volume_ - dynamics_[0]->volume_;

diff -Nru lilypond-2.14.2//lily/dynamic-performer.cc  
tmp//lily/dynamic-performer.cc
--- lilypond-2.14.2//lily/dynamic-performer.cc	2011-07-26  
01:42:15.000000000 +0300
+++ tmp//lily/dynamic-performer.cc      2011-09-11 17:24:08.000000000 +0300
@@ -111,7 +111,7 @@

   if (span_events_[START])
     {
-      span_dynamic_ = new Audio_span_dynamic ();
+      span_dynamic_ = new Audio_span_dynamic (equalize_volume (0.1),  
equalize_volume (1.0));
       announce_element (Audio_element_info (span_dynamic_,  
span_events_[START]));

       span_dynamic_->grow_dir_ = grow_dir_[START];
diff -Nru lilypond-2.14.2//lily/include/audio-item.hh  
tmp//lily/include/audio-item.hh
--- lilypond-2.14.2//lily/include/audio-item.hh	2011-07-26  
01:42:15.000000000 +0300
+++ tmp//lily/include/audio-item.hh     2011-09-11 17:20:36.000000000 +0300
@@ -54,11 +54,13 @@
 public:
   Direction grow_dir_;
   vector<Audio_dynamic*> dynamics_;
+  Real min_volume_;
+  Real max_volume_;


   virtual void render ();
   void add_absolute (Audio_dynamic*);
-  Audio_span_dynamic ();
+  Audio_span_dynamic (Real min_volume, Real max_volume);
 };

------

With this patch (applied to LilyPond 2.12.3), I was able to obtain
the result that I more or less expected from the example input (that is,
that the volume level of the accompanying instrument should not increase
noticeably beyond \ffff).  Here's the midi:

http://koti.welho.com/htauriai/test-2.12.3-patched.midi


Finally, reading about MIDI improvements in the new stable LilyPond 2.14
series released this year made me curious about what kind of an effect
the improvements might have on the default MIDI output produced from
this example.  First, here's the MIDI:

http://koti.welho.com/htauriai/test-2.14.2.midi

I think that the result produced by LilyPond 2.14.2 is, unfortunately,
(compared to my own expectations) even worse than with the previous
stable version.  In addition to the volume level restriction problem
(which seems to still go away when applying the above fix), it feels
like the volume of the accompaniment "chords" written as temporary
polyphony within a single voice is not affected at all by the changes
in dynamics.  This behavior seems new since version 2.12.

I suspect that this result could be somehow due to the new Voice
contexts introduced by the polyphony, if each of these contexts has a
Dynamic_performer of its own by default (as suggested in the Notation
Reference).  However, trying to move the Dynamic_performer to a
different context such as the Staff context did not help: after trying
    \midi {
      \context {
        \Voice
        \remove "Dynamic_performer"
      }
      \context {
        \Staff
        \consists "Dynamic_performer"
      }
    }
the only change that I was able to notice from listening to the MIDI
output was that now the absolute dynamic settings for the harpsichord
(\ppppp, \ffff) stopped taking effect immediately at the point of use;
instead, they started taking effect only at the next < > chord
_following_ the dynamic change.


In the end, getting all dynamic changes interpreted accurately in MIDI
is not that important to me, but ideally I would like to be able
to (1) reliably apply some form of equalization to the MIDI instruments
(for example, by specifying some initial volume levels, which would
then remain fixed throughout a score, independently of any temporary
contexts created within the score), and, on the other hand, to be
able to (2) transcribe scores without worrying, for example, about
adding extra absolute dynamic marks to prevent any surprises in
instrument volumes increasing or decreasing beyond specified limits
if the score happens to get rendered in MIDI.

Obviously, simply disabling the use of Dynamic_performers would
already be almost a solution for both problems, if only removing  
Dynamic_performers didn't also remove (to my understanding) the
possibility to apply equalization to MIDI instruments.  Therefore I
have the following questions:

- Is the failure of the Dynamic_performer to keep instrument volumes
  within specified ranges a known problem, or is my assumption that
  Staff.midiMinimumVolume and Staff.midiMaximumVolume represent
  absolute limits for MIDI volume (that is, that the volume of no
  instrument should ever be outside the volume range defined for the
  instrument using these limits) simply wrong?

- What about the behavior I observed with LilyPond 2.14 (volume levels
  within temporary Voice contexts appear to be unaffected by the volume
  levels of the temporally preceding music) -- why did this not
  occur with the previous stable version?  Are there possibly settings
  to make new Voice contexts "inherit" their initial dynamic settings
  from their enclosing contexts, or to make all dynamics for a single
  instrument to be interpreted in a common context (such as the Staff
  context)?  If so, what would be a correct way to do this with
  LilyPond 2.14?


Best regards,
Heikki Tauriainen