bug#18948: 24.4; CC mode causes jit-stealth-lock to signal an error

[Alan Mackenzie]

In article <mailman.12784.1415120784.1147.bug-gnu-emacs@gnu.org> you wrote:
> To reproduce, in "emacs -Q" evaluate the following:

>  (setq jit-lock-stealth-time 16)
>  (setq jit-lock-stealth-nice 0.5)
>  (setq jit-lock-stealth-verbose t)
>  (setq jit-lock-defer-contextually t)
>  (setq jit-lock-stealth-load 20)

> Then visit buffer.c from the current trunk, and wait for about 20 to
> 30 sec.  After a while, you should see the JIT Stealth messages
> blinking in the echo area, and after some more time, you will see an
> error message:

>  Error running timer `jit-lock-stealth-fontify': (args-out-of-range #<buffer 
> buffer.c> 71003 71397)

> This happens because some code in CC mode that handles fontification
> narrows buffer.c such that the accessible region ends at position
> 71145, inside a call to jit-lock-fontify-now that wants to fontify
> between the two positions shown in the error message.  Trying to
> access buffer positions beyond 71145 triggers the error.

Thanks for such deep debugging information, Eli!  This made it very easy
to track down the bug.

> The C and Lisp backtraces, both from the call to narrow-to-region
> which performed the narrowing, and the call to parse-partial-sexp
> which signals the error, are below.  AFAICS, the narrowing is done in
> this snippet from c-find-decl-spots:

>               (if (zerop cfd-macro-end)
>                   t
>                 (if (> cfd-macro-end (point))
>                     (progn (narrow-to-region (point-min) cfd-macro-end)
>                            t)
>                   ;; The matched token was the last thing in the macro,
>                   ;; so the whole match is bogus.
>                   (setq cfd-macro-end 0)
>                   nil))))

> The function then proceeds to call cfd-fun, which, several levels
> below, calls parse-partial-sexp that barfs.

At this stage, jit-lock-... has called, indirectly,
c-font-lock-declarations with a `limit'.  This calls c-find-decl-spots
with an internal (largish) lambda function as the `cfd-fun' argument.

c-find-decl-spots calls this lambda (`cdf-fun'), having narrowed the
buffer to the end of a macro.  The lambda then calls
c-font-lock-declarators with `limit' as argument, and this later leads to
the args-out-of-range condition.  Substituting (point-max) for `limit'
fixes it.

[ ... ]

> Curiously, I cannot reproduce this on the trunk, only with Emacs 24.4,
> but if that means the problem was somehow solved on the trunk, it
> would be nice to have that solution on the branch as well.

Also, the bug doesn't show up with Emacs 24.4's buffer.c.  Why it should
trigger so rarely is a mystery.  I suspect the answer has to do with the
"##" CPP operators in the vicinity, and that on the trunk's buffer.c, they
are triggering bug #18749 (where a "##" was spuriously recognised as a
macro start because of narrowing to exactly that point).

[ ... ]

Here's a patch which fixes it.  I'll get it committed to the trunk
sometime today.

Perhaps both this fix and that to 18749 should be backported to the 24.4
branch.


--- cc-fonts.el~        2014-03-21 05:34:40.000000000 +0000
+++ cc-fonts.el 2014-11-05 11:23:05.000000000 +0000
@@ -1507,7 +1507,7 @@
                        ;; At a real declaration?
                        (if (memq (c-forward-type t) '(t known found))
                            (progn
-                             (c-font-lock-declarators limit t is-typedef)
+                             (c-font-lock-declarators (point-max) t is-typedef)
                              nil)
                          ;; False alarm.  Return t to go on to the next check.
                          (goto-char start-pos)



-- 
Alan Mackenzie (Nuremberg, Germany).