Re: Last steps for pretesting (font-lock-extend-region-function)

[Alan Mackenzie]

Hi, Stefan!

On Tue, 25 Apr 2006, Stefan Monnier wrote:

>> I find the elisp manual page which describes font-lock-keywords
>> ("Search-based Fontification") all but inpenetrable.  It's reduced me to
>> tears on several occasions.  I've just spent five minutes looking through
>> it, trying to find the description of the form you suggested to me:

>>      (".*\\\\\n.*"
>>       (0 (progn (put-text-property (match-beginning 0) (match-end 0)
>>                                    'font-lock-multiline t)
>>                 nil)))

>> .  I didn't find it.

>This form is the standard form (except you haven't noticed yet that the
>FACENAME part of the font-lock-keywords is not a face symbol but an Elisp
>expression whose value after exavluation should be a face).  It's:

> (MATCHER HIGHLIGHT ...)

>where MATCHER is ".*\\\\\n.*" and HIGHLIGHT is of the form MATCH-HIGHLIGHT
>which is of the form (SUBEXP FACENAME [OVERRIDE [LAXMATCH]]) where SUBEXP is
>0 and FACENAME is (progn ...).

OK.  So FACENAME comes back as nil (which presumably means "don't apply a
facename property") and the side effect is applying f-l-multiline if the
buffer line matches the regexp.

>> With respect, Stefan, that's rubbish.  You need to know where and how
>> to write the above Lisp form (the one starting (".*\\\\n.*" ...)) into
>> f-l-keywords.  For me, that's perhaps half an hour to an hour of
>> frustration trying to find the right place in "Search-based
>> Fontification".

>If you don't know how font-lock-keywords work, then learn that first
>before trying to work on the font-lock support for cc-mode.  That'll
>save you a lot more time.

I cannot learn it, any more than I could learn logarithm tables when such
were still in use.  I need to go through the elisp page every single
time.  The same applies to things like c-offsets-alist too, by the way.

>> What about the run-time overhead of having an extra text property over
>> the entire buffer?

>It's not over the entire buffer.

Load a buffer and wait for it to be jit-locked.  The entire buffer will
have been scanned and this text property, in effect, set to t or nil on
every single byte.

>And I haven't seen any evidence that it's noticeable (especially
>compared the number of `face' properties or of the `fontified' property
>added to the whole buffer).

I'm sure it's not.

>> The f-l-e-r-f, as you envisage it (being called only from
>> f-l-default-fontify-r) will also be called at virtually every buffer
>> change, since almost every change causes a redisplay.

>AFAIK, the case where a single change takes place between redisplay is the
>case where there is no performance issue: it takes place at the user's
>pace anyway.  The important case is when Emacs has more work to do in
>response to a user's action, i.e. there's more work to be done until the
>next redisplay.

Is it really that important to responsiveness whether the delay happens
at the redisplay or immediately after the change?  With the after-change
f-l-e-r-f, only the two boundaries of the region get checked, plus any
chunk boundaries when this is bigger than 500 bytes.  With the extra
pattern in f-l-keywords, then entire region gets checked.

An after-change f-l-e-r-f needs to be fast enough, and if it's not, that
might be a good reason for using f-l-m instead.  However, f-l-m is more
expensive in aggregate - it marks _every_ sequence which must be
atomically fontified, just in case that sequence might straddle a chunk
boundary in the future.  Not only that, it erases and recalculates the
property every time it fontifies a region, even when there has been no
buffer change in that region.  After-change f-l-e-r-f only checks for
those region boundaries it needs to know immediately.

[ .... ]

>I'm as close as it gets to a font-lock maintainer.  So from where I
>stand, you're trying to impose your taste and judgment on me.

I'm as close as it gets to a CC Mode maintainer at the moment.  What's
constraining my freedom in that department now?  ;-)

[ .... ]

>> My fix is simpler: to recognise that f-l-default-fontify-region
>> performs two disparate tasks: (i) Extending the region to be
>> fontified; (ii) Fontifying this region.  By extracting (ii) into a
>> separate function, that function could be called directly from
>> jit-lock-fontify-now, bypassing the redundant second massaging of the
>> region.

>It has to be in font-lock-default-fontify-buffer since there's no
>guarantee this is only called from jit-lock-fontify-now.

No.  It has to be _called_ from font-lock-default-fontify-buffer.

>So you're suggesting to add a redundant call to the extend-region hook.
>Maybe it's simpler but it's ugly (because of the redundance, and because
>it adds a dependency between jit-lock and font-lock).

No.  I'm suggesting refactoring the font lock code a little so that each
path through it calls the extend-region hook exactly once.  Once in the
jit-lock "branch" and once in the plain font lock "branch".

>> (i) It uses the same hook, font-lock-extend-region-function, as
>> f-l-after-change-functions, setting the third parameter, OLD-LEN, to nil;

>> (ii) It uses a distinct hook with a new name, say,
>> font-lock-extend-chunk-function;

>> (iii) It usurps the name f-l-e-r-f from its current use, abolishing the
>> region extension mechanism in the f-l-after-change-f/j-l-after-change.

>(iv) it reuses the name font-lock-extend-region-function and renames the
>     a-c-f hook to font-lock-after-change-extend-region-function.

Or that.  It's a good job our terminals are wider than 80 columns
nowadays.  :-)

>This will happen hopefully today.

Hopefully yes!

>At the same time font-lock-after-change-extend-region-function will be
>moved from font-core to font-lock (where it belongs) and its
>make-variable-buffer-local call will be removed as well (use
>make-local-variable manually if you need it).

Surely not - that variable is essentially buffer local, and it makes no
sense for it not to be so.  To remove the make-variable-buffer-local call
could lead to the same problems which have bedevilled paragraph-start and
paragraph-separate.

>        Stefan

-- 
Alan.