Re: Default of jit-lock-stealth-time

[David Kastrup]

Eli Zaretskii <address@hidden> writes:

>> Cc: address@hidden,  address@hidden
>> From: David Kastrup <address@hidden>
>> Date: Sat, 03 Mar 2007 14:36:28 +0100
>> 
>> Apart from "pathological" buffers, paging through a file will deliver
>> font locking fast enough to follow the user action
>
> Where ``fast enough'' means what, precisely?
>
>> In contrast, left alone to jit-lock-stealth-time=16, Emacs will
>> eventually turn to eating 100% of CPU time (not 1-3%)
>
> How can this happen?  JIT-lock is supposed to fontify at most
> jit-lock-chunk-size characters (500 by default), and then refrain from
> fontification for at least jit-lock-stealth-nice seconds (0.5 by
> default).  How can Emacs take 100% of CPU with these defaults?

Actually, it is more around 75% of CPU time according to "top".
Enough to cause a power drain, keep Emacs mapped to memory, and
obviously in long enough bursts to keep the computer in general and/or
Emacs from being responsive.

> Maybe there's a bug?  How much time do you need to wait until Emacs
> starts using 100% of CPU on your machine?  I didn't see that, but
> maybe I didn't wait long enough.  Also, did you try looking at the
> percentage of CPU taken by each application, when the CPU
> consumption hits 100%, and if you did, was Emacs indeed consuming
> 100% or thereabouts at that time?

Since I just got hit again (I deleted my customization of
jit-lock-stealth-time after checking in the change, but did not
actually recompile here), I tried changing jit-lock-stealth-nice.
Changing it from the default of 0.5 to a value of 3 will cause the CPU
load that top reports Emacs to be taking (when it goes wild) to drop
from about 75% (for 0.5) to about 40% (for 3).  So a totally wild
interpolation would come to the conclusion that the "bad"
fontification attempts take about 75%*0.5sec/25% = 1.5sec or
40%*3sec/60%=2sec each, which is similar enough for such a rough
estimate.

Maybe stealth lock should keep track of the actual time it spends on
fontification and calculate its next jit-lock-stealth-nice value based
on that.  Or should disable stealth lock in buffers that seem not to
behave well.

As I said, the most common culprit at the moment for me seems to be
texbook.tex.gz.  I can send you a copy if you want to, but as I said,
the problem also seems to depend on some third-party packages.

After a few seconds of thrashing, I get the following results from
instrumenting font-latex:

Function Name                              Call Count  Elapsed Time  Average 
Time
=========================================  ==========  ============  
============
font-latex-match-command-in-braces         1780        3.2092149999  
0.0018029297
font-latex-find-matching-close             970         3.0349990000  
0.0031288649
font-latex-match-type-declaration          10          3.012727      0.3012727
font-latex-match-simple-command            62693       2.2595180000  
3.604...e-05
font-latex-script                          14685       1.3847209999  
9.429...e-05
font-latex-match-command-with-arguments    4800        1.2166909999  
0.0002534772
font-latex-match-script                    14695       1.1428459999  
7.777...e-05
font-latex-match-function                  4210        0.8861020000  
0.0002104755
font-latex-commented-outp                  27908       0.7627639999  
2.733...e-05
font-latex-unfontify-region                10          0.4944489999  0.0494449

Now 62693 is a large number, and the whole file apparently contains
just 37633 occurences of LaTeX commands.  The stealth fontification
for which I have now profiled just a few seconds, pretty much goes on
indefinitely.  So it would appear that there is something like
quadratic behavior in some circumstances triggered in connection with
stealth fontification.

So yes, there is likely a bug (or whatever one may call something that
takes up too much time) in this particularly severe case, but it is
not triggered outside of stealth fontification, and stealth
fontification makes the problem be pretty much impossible to debug
properly, and triggers it when editing _any_ buffer.

I can send you the file in question, but you'll probably need
font-latex and possible other AUCTeX components for reproducing the
problem.  But it is likely to be a black hole: there are other modes
with possible font locking problems, and it is just not a good idea to
spread their effects across a whole Emacs session.  That way, they
will much less likely be fixed some day.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum