Re: font-lock-extend-region

[Stefan Monnier]

[ I hope some of this exchange can get into some documentation somewhere.
  I find it very useful to have to explain it in such details. ]

> Before I get going, I'd like to say I've spent some time getting to grips
> with jit-lock, and I think I now understand some of the things you were
> telling me.  I also apologize for getting a bit grumpy about it last
> week.

Don't worry about it.  I've been so edgy these days that I wouldn't notice
anyone else getting grumpy.

> I think there are two distinct issues here that we're confusing, and this
> is why we've found it so hard to agree:

>     (i) calculating the region which needs refontifying.
>     (ii) finding a safe place to start fontifying a single chunk.

Right.  The first is generally handled by jit-lock-context-* (including the
jit-lock-defer-multiline property).  The second is generally handled by the
font-lock-multiline property and by rounding up to a whole number of lines.

> font-lock-extend-region-function is intended to do (i).  The functionality
> you're suggesting for f-l-default-fontify-region is for doing (ii).

In other words your use of font-lock-extend-region-function is specifically
to deal with issues that jit-lock-context-* tries to handle as well.

> I think the essence of the font-lock-multiline property is that it marks
> a chunk of text to be fontified atomically.  Please confirm this
> impression or correct it for me.

Correct.

> Here's why I think the font-lock-multiline way is wrong.  Taking my AWK
> example again:

> 1. "string \
> 2. over \
> 3. several \       <=========
> 4. #lines."

> Suppose the user replaces the backslash on L3 with 20k of code from the
> kill ring with M-y.  The region to fontify now extends from L1 to EOL4
> (actually, it's now L1073).  The display engine is going to request
> fontification from L1034.  If I mark this entire region with
> font-lock-multiline, these 1073 lines will be (unnecessarily) fontified
> atomically, defeating the aims of jit-lock in this case.

The region is not automatically marked with font-lock-multiline, so you
can't really fault font-lock-multiline for it: it's your code that marks it
that is at fault.
[ Now don't get me wrong: the font-lock-multiline property is not perfect. ]

> What I think we need is a function called from f-l-default-f-region which
> will get a safe starting position at or before L1034.

Agreed.  And I suggested we name it font-lock-extend-region-function.

You seem to be saying that you'd also want such a thing in
after-change-functions.

My belief is that you don't need it for the following reason: if you need to
refontify more than the 20K of code you just yanked, it can only be because
of elements at the beginning/end that need to be refontified atomically, so
you can just either place a font-lock-multiline property on them or extend
the region from f-l-default-fontify-region.

But maybe this is only true in theory, and reasons of performance (or
presence/absence of various info in different contexts) make it that you do
need an "extend-region" in font-lock-after-change-function.  Is that what
you are saying?

Or is there some other reason to extend the region from
after-change-functions, other than atomic elements at the boundaries?

> What I think we should do is to put a hook into f-l-default-f-region to
> calculate a safe starting position (and probably also a safe stopping
> position).

Yes, we agree on that.

> Incidentally, referring to my diagram above, the region gets extended to
> whole lines more than once.  For demand fontification, it is done first
> in jit-lock-fontify-now then in font-lock-default-fontify-region.  For
> after-change fontification, it is done yet a third time in
> jit-lock-after-change.

Yes, it's a bit messy, partly for historical reasons.

> How about doing this only in f-l-default-f-r?  This would make it easier
> for a mode maintainer to switch off this action, since he would then just
> have to put a modified function into the hook
> font-lock-fontify-region-function.

I believe the one in jit-lock-fontify-now could be removed (but this
function is also sadly called from external packages, so there may be some
minor compatibility issues).

But the one in jit-lock-after-change is needed because of what the comment
there says.  Basically here is the scenario:

  start with C code like

    void foo 

  and add an open parenthesis at the end.  The modified chunk is just the
  open paren, so if you don't reset the `fontified' property on the whole
  line, the redisplay engine will not redisplay `foo' and even if jit-lock
  changes the `face' property on `foo' it does it after the display engine
  decided what `foo' would look like.  So if jit-lock-after-change doesn't
  round up to whole lines, `foo' in the above scenario would only be
  refontified at the next screen refresh :-(

I'd like to be able to solve this problem elsewhere than in
jit-lock-after-change (e.g. some way for jit-lock to say "hey, font-lock
modified this text before BEGIN, please make sure you redisplay it
immediately"), but even if I knew how to do that, I'd probably not use it
for whole-line-round-up because it would simply cause (almost) all redisplay
to be done twice.


        Stefan