Re: address@hidden: C++-mode: Syntax highlighting: wrong color for function identifier depending on the kind of whitespace that follows]

[Stefan Monnier]

>> Also, I thought we had agreed that this new feature should be in
>> f-l-default-fontify-region rather than f-l-after-change-function.  I believe
>> it is *wrong* for it to be in f-l-after-change-function where it is mostly
>> useless:

> It actually works quite well, I've tested it.  :-)

Please re-read what I wrote.  I don't say it won't work.  It has various
problems, tho, which you most likely wouldn't notice in random testing,
especially if you turn off jit-lock.

> I'm convinced this feature needs to be in the font-lock/jit-lock
> after-change-functions.  The fontification region is determined by the
> _change_, not merely by the buffer contents after the change.  For
> example:

> The stealth/display fontification would extend their regions using these
> functions,

Thinking of stealth fontification as a special case will not lead you to
a good solution.  Been there, done that.  All the problems I've ever
encountered that were supposedly "due to jit-lock" could also be reproduced
without jit-lock.

>> you can already get the same result by using an
>> (non-f-l)after-change-functions hook that sets the
>> font-lock-multiline property.

> OK, but for that, font-lock-multiline must first be documented.

Go for it.  Note that I'm not talking about the variable, but about the
text-property.

> I've tried to figure out from the source how f-l-multiline works, but
> haven't succeeded yet.  Can f-l-multiline be used to extend the
> fontification region backwards?

Of course.  Put font-lock-multiline property on all the chars of a region
that should be refontified as a whole because the font-lock-keywords don't
know how to refontify it correctly unless they get to see the whole thing
at once.  f-l-default-fontify-region then extends the region
forward&backward so the region it refontifies doesn't start or end in the
middle of a font-lock-multiline property.

> With the f-l-multiline approach, you need somehow to ensure that the
> non-f-l-after-change function gets called before f-l-after-change.

Not if you use jit-lock ;-)

> It can be done, but is hassle.  Also, f-l-multiline is an internal Font
> Lock mechanism, and major modes shouldn't have to manipulate it directly,
> since it would increase the coupling between modes and Font Lock
> enormously, making it expensive to change approach in the future.

The feature you added suffers from the exact same problem.

> #########################################################################

>>> More controversially, I've explicitly documented that the region returned
>>> by the f-l-extend-region may start or end in the middle of a line.

>> I don't care much either way.  If someone does that, it's his problem:
>> it won't hurt font-lock.el.  But it may screw up the other
>> font-lock-keywords.

> Why would it be a problem?  Given that f-l-extend-region-function and
> font-lock-keywords both belong to the major mode, they can be made to
> match eachother.

Exactly: the author can only shoot himself, so "it's his problem".
Except of course for minor modes which add their own keywords (via
font-lock-add-keywords, for example), but these already suffer from several
other comparable minor problems, so it's probably not making things
noticeably worse.

> A comment in jit-lock says that if the f-l region starts in the middle of
> a line, that line won't be re-displayed.

Which comment?

>>> The "baz" on L3 hasn't a snowball's chance in Hades of getting fontified
>>> with f-l-function-name-face unless the font lock region is allowed to
>>> start at the "int" on L2.

>> Why?  After all, when you open the file, the region will start at BOB.

> When you open the file, yes.

> (i) You type a space on L2, "baz" changes to variable-name-face.

> (ii) You then type a space on L3, it flips back to function-name-face,
> but "bar" goes to default face.  (font-lock-lines-before == 1)

> (iii) Type a space on L1, "bar" is restored to variable-name-face after
> ~0.5 second (contextual fontification?).

> (iv) reload the file, do (i), then put point over "baz" (currently
> variable-name-face).  Do C-u C-x =, to display its properties.  It
> changes instantly to function-name-face, which it reports in the other
> window.  Hello Heisenberg!

I dno't see where the above example requires the region to start "at the int
on L2" (or more generally in the middle of a line).

> I honestly don't think it's worthwhile to investigate each of these
> happenings individually - it's amusing to watch for a few minutes, but
> rapidly becomes tedious.  They all stem from a single cause, namely the
> misapprehension that font lock can start analyzing code at an
> arbitrary BOL.

It's not a misapprehension: it's a constraint.  If your font-lock-keywords
don't obey this constraint, then yes, you'll see funny things because your
font-lock-keywords are broken.  Lifting this constraint is what "adding
multiline keywords support" is all about.  But I still haven't seen any
example where it needs to be possible to start in the middle of a line.
What is often neded is to be able to prevent starting/ending at some
particular BOLs.

> OK, these are likely to be bugs in CC Mode's font lock patterns, but the
> uncertainty caused by font-lock's instability makes them more difficult
> to debug.

Easy: turn off jit-lock and it becomes much more deterministic (and those
bugs can still be easily reproduced).  Sometimes you just need to cut&paste
a block of text at the same spot (i.e. mark and then C-w C-y).

> (i) f-l-extend-region-function should be supplemented by
> f-l-safe-\(floor\|ceiling\)-position-function; if non-nil, these
> functions give guaranteed good boundaries for fontification.  They would
> render contextual fontification redundant.

No, it's still not redundant at all.  Contextual fontification is solving
other cases of multiline patterns where the approach of extending the region
is unworkable (because the region may end up extended much too far, bringing
Emacs to a crawl): e.g. removing/adding a " in a C file typically causes the
whole rest of the buffer to be refontified.

> (ii) Code expanding the font lock region to whole lines should be
> removed (except as in (iii)).  font-lock-multiline would become
> redundant.

I fail to see the conection between font-lock-multiline and whole lines.

> (iv) The interface between Font Lock and the display code should be
> amended so that a font lock region need not start at column 0.

I don't think there is any such requirement.


        Stefan