Re: Lilypond taking forever to typeset

[Mojca Miklavec]

Hi everyone,

Just to explain a bit more.

OS X has its own font libraries, but now that a lot of free software
is coming from Linux world and without extensive effort to try to get
more "mac-friendly", those software packages now use fc-cache (and its
equivalents) on Windows and OS X as well. However, in contrast to
linux where there is only a single intance of fontconfig, on OS X (and
windows) each and every software comes with its own copy of fontconfig
and needs its own font library cache.

I'm experiencing the same "problem" with many different software packages:
- gnuplot (when using cairo-based terminals)
- mplayer (I have two variants of it)
- VLC
- I believe that XeTeX needs fontconfig on Windows, while it uses the
system libraries on OS X
- LuaTeX (ConTeXt MkIV and luaotfload) has its own implementation for
creating the caches
- ...
just to name a few.

See also
    https://sourceforge.net/p/gnuplot/mailman/message/34852862/
and feel free to ignore my feedback as it might be biased and
expressing the same opinion as I provide here.

(Note that the person compiling windows binaries only experienced the
delay on one binary, but not on the other. This might be due to a
slightly different configuration of fontconfig, I don't know.)

VLC and some variant of mplayer's GUI warn the user saying "generating
font database" and show some kind of a spinning wheel, so that the
user at least knows that the software didn't hang and patiently waits.
On the other hand I was sure that my lilypond was hanging and
reinstalled it.

But I don't remember any of those software packages to actually take
the whole 10 minutes to do the font caching.

Please also note that my laptop is from 2009, without SSD etc., so
most computers would finish processing much much faster.

To answer to David: yes, it's one-time cost (that gets re-triggered
every now and then in order to be able to get a more recent database
including potential new fonts on the system, but I don't know the
"rule" of when this gets triggered). In any case, after the initial
10-minute-waiting each lilypond processing is reasonably fast.

In order to be able to reproduce the problem reliably I would have to
figure out where the font cache is stored and make sure to delete it.

There is a folder
    LilyPond.app/Contents/Resources/var/cache/fontconfig
but it is empty (which makes sense).

There is
    
~/Library/Caches/org.lilypond.lilypond/com.apple.opencl/com.apple.ocl.32.{data|maps}
which seems to be something else.

I'm not yet sure where else to look for.

On 11 July 2016 at 12:17, Werner LEMBERG wrote:
>
>> So if you have some manner of reproducing the problem it would be
>> nice if you could check against something like version 2.19.40 so
>> that we can at least figure out whether it is the most recent batch
>> of changes which is responsible here.
>
> Yes, we need that.  A quick look at the recent font handling changes
> doesn't show up a modification that would make lilypond suddenly
> iterate over all system fonts.
>
> Please provide a minimum example.

I'm unable to provide a minimum example. I'm 99% sure that the long
delay is independent of the input. (I have tried several different
examples.) I only realized that it might be a problem with font caches
because I checked which files lilypond had opened and noticed that it
was iterating through system fonts, spending several seconds per font.

> Do you have time to bisect the problem?

Do you happen to have an archive of OS X binaries?

In any case I have to figure out how exactly to test before I even
start testing. I could do some testing with MacPorts (I could easily
compile from source, I only have to invest the initial time to compile
all the dependencies), but I'm afraid that it might not be
reproducible there because there might be a single fontconfig database
for the whole MacPorts installation. I have to think a bit and figure
out what to do and how to test.

I don't understand why the problem doesn't appear on Windows though (I
didn't test lilypond on Windows, but I would imagine that many other
users would complain).

Mojca

PS/unrelated: Talking about macs, I don't like the fact that I have to
create a script doing
    #!/bin/bash
    exec /Applications/LilyPond.app/Contents/Resources/bin/lilypond "$@"
Why doesn't simple symlinking work? If I try symlinking
    /Applications/LilyPond.app/Contents/Resources/bin/lilypond
here's what I get:

GNU LilyPond 2.19.45
warning: not relocating, no 2.19.45/ or current/ found under
/Users/me/share/lilypond/
ERROR: In procedure primitive-load-path:
ERROR: Unable to find file "ice-9/boot-9.scm" in load path

There is an easy workaround of course and it's not a problem for
anyone reading the docs, but it's not intuitive and it would be better
to fix it at some point.