RE: Error message

[Andrew Bernard]

Hi All,

I have been working closely with Craig to study the matter. Interestingly,
in this large incomplete score as received, it consumed an ever expanding
amount of memory until it crashed with no output. I observed a set of
warnings regarding missing parts. In my experience one ignores lilypond
warnings, even though not errors, at one's peril. I asked Craig to address
the errors, and then it compiled fine, although still consuming a lot of
memory. That in itself is interesting.

I ran on Linux Mint 18.1, Windows 10, and Mac OS X 10.12.5. All work fine
now. The remaining issue is that as Craig needs to add yet more parts to
this large wind orchestra work that it will max out memory again and fail.
We have yet to see.

Memory use for the lilypond process reaches about 4.9 GiB on linux, and
about 2.9-3 GiB on Windows and OS X.

Looking at the source code, it is clear that Craig is a master user of
lilypond, and I could not find anything obviously funky or misguided or
peculiar in the code that may cause memory blowouts.

My conclusion is that lilypond at this point in time does not handle large
scores in a constrained manner re memory, and as the score gets larger and
longer and longer, so lilypond's memory use just increases in linear
proportion. There must not be any pipelining and parallelism going on.
Indeed, I believe a very closely related thread regarding graphics
pipelining is current in the development list. I can say from this study
here that improvements in this area would welcome, nay, necessary for
ambitiously large projects. If this score reached 200 hundred pages, I doubt
it would compile on 32 bit systems, based in my measurements. And as
mentioned, we fear that adding the remaining parts will require more memory
than is available on 32 bit machines.

Regarding memory usage, I posed a question on the devel list which has so
far been overlooked. Why is the Mac OSX version of lilypond 32 bit, when
Macs have been6 4 bit for a long time? The same question can be asked for
Windows. Why is lilypond on Windows also only 32 bit? Is there some
technical limitation forcing this? Or is it just that the average use does
not make vast scores?

I will mention that it is interesting that on Windows the ram does sit just
under the 3 GiB limit imposed on processes by Microsoft, so perhaps lilypond
is behaving nicely and the linear increase in memory use with size is not
actually correct, and lilypond is managing memory, up to the 32 bit system
limit on Windows, and that it was unaddressed warnings blowing memory out
for some reason.

Andrew