Re: Last steps for pretesting (font-lock-extend-region-function)

[Stefan Monnier]

>> IIRC part (i) needs more than an after-change-function: it also needs
>> a before-change-function.

> That would depend on the major mode, I think.  AWK mode needs a
> before-change function.  The CC Mode replacement for (setq
> font-lock-lines-before 2) will not need a b-c function.

Simulating font-lock-lines-before only needs an `extend-region' hook in
font-lock-default-fontify-region, a subject about which we already agree.

> However, using the f-l-multiline property approach instead will also need
> a before-change function for exactly the same reasons

That's news to me.  Could you explain the scenario you have in mind?

> - or else it will require the major mode to maintain the f-l-multiline
> property permanently.

The font-lock-multiline property is removed by font-lock-unfontify-region
and it is expected that it gets re-added (if needed) every time you fontify
a piece of text.  It's never seemed to be a big deal.
No before-change-function in sight.

>> And (again IIRC) it's the part that can already be solved by placing
>> (from one of the font-lock-keywords rules) a font-lock-multiline
>> property.

> I'm not sure.  Can a font-lock-keyword rule get access to after-change's
> OLD-LEN in an reasonable fashion?

What for?  The font-lock-multiline it needs to add is for use by the next
font-lock round.  It says: "next time you fontify this area, make sure you
refontify this piece atomically".

> If a change consists of yanking 20k of text into a buffer, the Major Mode
> must not splat font-lock-multiline onto 20k + 20 bytes, since this would
> defeat the purpose of JIT lock (fontifying a bit at a time).

The font-lock-multiline property is added not to the whole fontified text,
but just to a particular region of text that matched a pattern (a
"keyword").  I think you're absolutely right: I have to document it because
you seem to completely misunderstand it.

> Could you please document the font-lock-multiline mechanism immediately,
> including details on how best to access the after-change-functions's
> OLD-LEN, and details of how and where to hook in the "(put-text-property a
> b 'font-lock-multiline t)" call.  Either that, or tell me where to find
> an example of this text property being used, preferably one which uses
> OLD-LEN.

AFAIK the OLD-LEN is never needed.  See font-lock.el for an example of code
that sets font-lock-multiline property: if a regexp matches several lines,
it puts the font-lock-multiline property on the wholse matched text (this
is only done if the font-lock-multiline variable is set).

> Or, perhaps you could recode the following AWK Mode fragment into an
> equivalent which uses f-l-multiline instead of f-l-expand-region
> function.  PLEASE!

I'll try to, although I'd need to see the font-lock-keywords part (where
I'll have to add the code).

> #########################################################################
> (defvar c-awk-old-EOLL 0)
> (make-variable-buffer-local 'c-awk-old-EOLL)
> ;; End of logical line following the region which is about to be changed.  Set
> ;; in c-awk-before-change and used in c-awk-font-lock-extend-region.

> (defun c-awk-before-change (beg end)
> ;; This function is called exclusively from the before-change-functions hook.
> ;; It does two things: Finds the end of the (logical) line on which END lies,
> ;; and clears c-awk-NL-prop text properties from this point onwards.
>   (save-restriction
>     (save-excursion
>       (setq c-awk-old-EOLL (c-awk-end-of-logical-line end))
>       (c-save-buffer-state nil
>        (c-awk-clear-NL-props end (point-max))))))
> (add-hook 'before-change-functions c-awk-before-change nil t)

> (defun c-awk-end-of-change-region (beg end old-len)
>   ;; Find the end of the region which needs to be font-locked after a change.
>   ;; This is the end of the logical line on which the change happened, either
>   ;; as it was before the change, or as it is now, whichever is later.
>   ;; N.B. point is left undefined.
>   (max (+ (- c-awk-old-EOLL old-len) (- end beg))
>        (c-awk-end-of-logical-line end)))

> (defun c-awk-font-lock-extend-region (beg end old-len)
>   (cons (c-awk-beginning-of-logical-line beg)
>         (if old-len
>             ;; From an after-change-function
>             (c-awk-end-of-change-region beg end old-len)
>           ;; From a fontifying routine
>           (c-awk-end-of-logical-line end))))
> (setq font-lock-extend-region-function 'c-awk-font-lock-extend-region)
> #########################################################################

IIUC, something like the following should do:

(defun c-awk-font-lock-extend-region (beg end)
  (cons (c-awk-beginning-of-logical-line beg)
        (c-awk-end-of-logical-line end)))
(setq font-lock-extend-region-function 'c-awk-font-lock-extend-region)

(defconst c-awk-font-lock-keywords
   '(...
     (".*\\\\\n.*"
      (0 (progn (put-text-property (match-beginning 0) (match-end 0)
                                   'font-lock-multiline t)
                nil)))
     ...))

although I suspect that you already have a font-lock-keyword that matches
something akin to ".*\\\\\n.*" where you can add the put-text-property.

>> Here is the problems I see with your proposal:
>> - an `extend-region' hook in font-lock-fontify-region is needed

> It is indeed.  It is a simple patch, and I am ready and willing to write
> it, both for font-core.el and modes.texi.  I do first ask that it's
> accepted in principle, though.

It's completely accepted.  I just hope we can call it
"font-lock-extend-region-function".

>> - the font-lock-multiline property should be enough in all cases to make
>> it unnecessary to use an after-change-function hook.

> f-l-extend-region-function also makes a (separate) after-change-f hook
> unnecessary.

Which f-l-extend-region-function?  The one called from after-change-f?
Then it's what I meant by "an after-change-function hook".  The fact that
it's not called directly from a-c-f but via f-t-a-c-f is not very
important for this discussion.

>> - an after-change-function is expensive, and even more when it comes bundled
>> with a before-change-function, and even more so when compared to
>> code that's only run during font-locking.
> Huh?  Font Locking _is_ expensive.

Yup, but (thanks to jit-lock) it's not executed after each and every
buffer modification.

> A before-change function need not be expensive (e.g.  c-awk-before-change,
> see above), and calling it is not expensive.

The code will have probably the same cost whether it's called from a-c-f or
from font-lock, but in one case it'll be called (much) more often.

> I suspect that using the f-l-multiline property will absolutely mandate an
> extra after-change function - how else can it access OLD-LEN?

Why would it need OLD-LEN?


        Stefan