Re: [Denemo-devel] GSoC proposal to improve MIDI and audio capabilities

[Dominic Sacré]

Hi Richard,

On Thursday 07 April 2011 18:45:58 Richard Shann wrote:
> On Thu, 2011-04-07 at 15:07 +0200, Dominic Sacré wrote:
> > I didn't mean to imply that Denemo is full of broken features, just
> > that my proposal is mainly about restructuring and cleanup, not
> > about removing things.
> 
> yes, I didn't take offence, I understood that, my question is would
> the re-structuring you propose allow the adjustment of the MIDI
> timing during feedback, as exemplified by the Tempo slider. That is
> if you could describe the sequence of events that would happen when
> someone moves the tempo slider during playback then I would be more
> confident that we weren't getting ourselves into a trap.

I'm not sure if I can describe the sequence of events that would happen, 
as I'm still struggling to understand how the current codebase works, or 
more importantly why it was written the way it is.
What I can tell you is that things like the tempo slider would just keep 
working like they do now. With the changes I'm proposing, during 
playback all MIDI events would still be read one by one from the SMF 
structure (see fluidsynth_play_smf_event() and 
jackmidi_play_smf_event()). The difference would just be that currently 
it's up to FluidSynth to create its own output ports (or access the 
soundcard, depending on the underlying audio API), while I'm proposing 
to do that by ourselves in Denemo. Nevertheless, the way MIDI events are 
fed from the SMF structure to the FluidSynth engine would remain more or 
less the same.

That said, there's of course also the timing issue that Philipp 
described. Now that I'm thinking about it, this is probably what 
prompted the suggestion of "making midi/audio output realtime-capable" 
as a GSoC project. Having written some JACK clients I automatically read 
that as "make JACK support realtime-safe", but bad MIDI timing is 
actually a different issue that has nothing to do with realtime-safety 
per se.

As far as I can see, both the FluidSynth and JACK MIDI backends use 
g_idle_add() to queue MIDI events for playback. It's pretty obvious that 
timing-wise this is a terrible idea. Could anyone explain why the code 
was written that way?
A proper multi-threaded model seems to be the way to solve this, and 
again, Philipp explained this much better than I did (in fact, I didn't 
mention in at all...). I still believe the best approach is to tackle 
all these problems at once in one major overhaul.

> > It has also been suggested that audio input is only semi-usable
> 
> I use it every week to tune my harpsichord. But I have not heard
> reports of people using it for inputting notes to Denemo - it
> certainly still does that - due to noises in the room I usually get
> a few notes inserted before I hit the Tuning button - but it will
> surely have some bit rot by now - it has lots of fancy pattern
> following features ... (I got a MIDI keyboard a couple of years ago
> and have never looked back).
> 
> > and
> > should not be a high priority. I'm not so sure about that, and
> > would rather fix it than remove it, as it's a pretty nifty
> > feature. Incidentally, I haven't been able to test it yet. At the
> > moment all my usable hardware audio inputs are JACK-only
> > (firewire), and audio input in Denemo only works with PortAudio.
> > Did I mention I was planning to fix that? ;)
> 
> yes you did, and I only mentioned that if you needed to cut corners
> to achieve the goal audio input could be left untouched, it would
> not break anything and would not be broken if it was left untouched.

Fortunately, removing the PortAudio dependency from the audio input code 
doesn't look very difficult. It may even be easier to just do that, 
rather than working around this part of the code just to be able to 
leave it untouched, while the rest of the audio/MIDI I/O code is turned 
upside down...

As for cutting corners, in the latest (final) version of my GSoC 
proposal I de-emphasized support for Windows and OS X. I wouldn't 
consider my work finished until those are supported (as they are now), 
but building and testing Denemo on all these different platforms is a 
can of worms I'd rather not open while I'm on such a tight schedule.


Best regards,

Dominic