Keep parsed document in memory (was: LilyPond as a service)

[Urs Liska]

Hi David,

thank you for your explanation, which is the base for a (presumably more
complicated) follow-up question.

Am 17.02.2016 um 23:31 schrieb David Kastrup:
> Urs Liska <address@hidden> writes:
>
>> I know this has been discussed in various flavors over the years, but I
>> need to raise the issue once more. Please note that this actually
>> encloses two questions.
>>
>> Would it be possible to make LilyPond work in a server mode where the
>> whole Guile environment is loaded only once and the running server would
>> be listening and accepting files to compile (or maybe piped .ly input or
>> even Scheme music expressions)?
> Shrug.  LilyPond already compiles multiple independent files/sessions
> per invocation or our documentation would need even longer to build.
>
> It's not really involved.  You just need to replace/amend the file
> handling loop in lily.scm.  See how -dgui is implemented in that file.

Thank you. I will go there and look..

>> How fundamentally would one have to turn LilyPond's architecture
>> upside down for this? Is this conceptually impossible, just difficult
>> or simply too big to have been tackled so far?
> Nobody cared.

I see. Maybe it's also because the default use case doesn't seem to call
for such a thing.
Although I think that as we have a default use case (tweaking layout)
that works on iterations this should be helpful (I mean if an IDE keeps
LilyPond running as a server and manages recompilation through this it
might as well (significantly) improve user experience.

>
>> Is this a task that could *somehow* be estimated in developer
>> weeks/months if one would dream of a paid developer?
> A few days for the proof-of-concept implementation once you find someone
> interested in sockets etc.  The actual effort, like with almost any
> functionality, is not limited.  There is always one more thing you want
> done.

Of course ;-)

> ...
>

So, let's assume we already had implemented a server daemon mode for
LilyPond.
Would it be possible to make that daemon keep a representation of the
*parsed* document in memory?

OK, looking at it from traditional use-cases this might seem
nonsensical. Why would you want to recompile an unchanged document?
Well, if I'm not mistaken the engraving can still be influenced *after*
the parsing by applying tweaks and overrides through engravers. I'm
concretely talking about the edition-engraver but eventually a newly
developed tool based on the current edition-engraver. And if I'm able to
do that *and* keep the parsed document in memory I could then recompile
the score or excerpts from it (using skipTypesetting) without
re-parsing. As most tweaks in the finishing stage of a project will only
affect the current system this would mean the recompilation can

  * skip the Guile set-up
  * skip the document parsing
  * engrave only a single system

This set-up can be managed by an editor that displays the score as a
sequence of single-system files, and this should actually result in
near-instant updates upon tweaking! (and we're also talking about
creating a graphical interface to generate the edition-engraver tweaks.)

So, does this seem feasible at all (that is: *is* there actually
something like a representation that could be kept in memory)? Does it
seem feasible when talking about one full-time developer for a certain time?

Any ideas?
Urs