Re: aux files to cache compilation info

[address@hidden]

On 7 août 2012, at 04:41, Han-Wen Nienhuys <address@hidden> wrote:

> On Mon, Aug 6, 2012 at 4:36 AM, Ramana Kumar <address@hidden> wrote:
>> In this message on users
>> http://lists.gnu.org/archive/html/lilypond-user/2012-08/msg00094.html
>> Mike mentioned the possibility of generating cache files to speed up
>> recompilation. He also said working out the details might take months
>> of discussion. I'd like to start that discussion in a relaxed way (we
>> have no deadlines after all), at least to generate an archive of
>> issues relevant to such a project. (Let me know if this has been
>> discussed before.)
>> 
>> People seem to agree that line breaking is the major bottleneck in
>> compilation. What would it take to cache just line breaking?
>> 
>> I suggest there are at least (and probably exactly) two things the
>> cache needs to contain:
>> 1. enough information about the last score compiled to know where the
>> new (current) score differs
>> 2. line breaking information for the cached score
>> 
>> Given that information, a possible use of the cache is: reuse line
>> breaking for any stretch of unchanged score where the line breaks
>> happen to be in sync (this always at least includes the stretch from
>> the beginning up to the first difference).
>> 
>> Thoughts on how to represent 1 and 2 above, and how to use them (to
>> find the stretches in sync)?
> 
> I'd forget about the stretches, and do it at a whole score granularity.
> 
> Dump a list (moment, breakable?) list as a key; if someone adds a note
> anywhere, you have to recompute everything. Later you can refine this;
> for the proof of concept it does not have to be more advanced, and
> this already lets people quickly iterate on fixing small formatting
> details.
> 
> For the line breaking, just dump a list of page/line breaks, ie. a list of
> 
>  (moment, is-page-break?)
> 
> one entry for each line-break. You should be able to plug this in to
> the code rather straightforwardly. If you only have to do it once per
> line, the spacing computation is actually pretty cheap, so you get
> most of the benefit with just this information.
> 
> You can use Scheme data structures as a data format.
> 
> -- 
> Han-Wen Nienhuys - address@hidden - http://www.xs4all.nl/~hanwen
> 
> _______________________________________________
> lilypond-devel mailing list
> address@hidden
> https://lists.gnu.org/mailman/listinfo/lilypond-devel

I did my first tests with this on a Mozart horn concerto that takes 42 seconds 
to compile.  It looks like using .aux files would shave off 2-4 seconds for the 
line breaking (assuming that "line breaking" is the time it takes to execute 
the function get_line_forces in Simple_spacer - there's obviously more to it 
but this is the workhorse) and 3-4 seconds for page breaking if using 
ly:optimal-breaking.  So even with aux files, I'm not sure if this would be a 
valuable enough time saver to implement.

I think that the best use of development time w/ regards to this issue would be 
the following:
1) Optimize the current code base (which we do anyway).
2) Create an ugly-lily.ly file.  This file would remove several engravers, 
change most callbacks to hardcoded values and have everything spaced on one 
line unless breaks were forced and would output the document to a huge page.  
This could be used for people doing mockups and then removed to get the nice 
score.

Cheers,
MS