Re: [Emacs-diffs] emacs-24 d69e9f1: CC Mode: Stop Font Lock forcing fontification from BOL. Fixes debbugs#19669.

[Alan Mackenzie]

Hello, Daniel.

On Fri, Mar 20, 2015 at 02:25:09PM -0700, Daniel Colascione wrote:
> On 03/20/2015 10:29 AM, Alan Mackenzie wrote:
> > On Fri, Mar 20, 2015 at 09:34:52AM -0700, Daniel Colascione wrote:
> >> On 03/20/2015 09:30 AM, Alan Mackenzie wrote:
> >>> On Thu, Mar 19, 2015 at 04:56:16PM -0400, Stefan Monnier wrote:
> >>>>>> you can handle fontification from 1,3, or 4 but not from 2.  You need 
> >>>>>> to
> >>>>>> refine the system you use to keep track of whether we're within
> >>>>>> a `union' so that it knows that position 2 is also "within a union".

> >>>>> Well, sort of.  The problem I'm facing is that in Dima Kogan's bug
> >>>>> #19669, the following construct appears:

> >>>>> 1. enum xxx_xxxx_xxxxxxxxxx_x
> >>>>> 2.     {XXX_XXXXXX_XXXX_XXX,
> >>>>> 3.      XXX_XXXXXX_XXX_XXX,
> >>>>> 4.      XXXX_XXXXX_XXXX_XXX,

> >>>>> Note that the brace on L2 is on the same line as the first XXX_....

> >>>>> When the user types on line 4, 5, ... here, CC Mode sets the
> >>>>> fontification region start to JUST AFTER THE { ON L2.  It is essential
> >>>>> that Font Lock doesn't change this.

> >>>> There's your problem: your current setup needs the starting point to be
> >>>> either before "union" or after the first open brace.
> >>>> It breaks down if it's between the two.  That's the problem you need to 
> >>>> fix.

> >>> Not really.  CC Mode is quite capable of handling the Font Lock region
> >>> starting at BOL2.  The problem is, when that starting point needs to be
> >>> after the brace on L2, Font Lock moves the starting point somewhere
> >>> else, fouling up the font locking.  This is proving surprisingly tough
> >>> to fix.

> >> I don't understand. Why shouldn't I be able to tell cc-mode to fontify
> >> *arbitrary* regions and expect it to correctly apply highlighting to
> >> these regions?

> > You can, as a CC Mode user.  In its turn CC Mode needs to be able to
> > select the region in which it does its analysis, so that that analysis
> > starts at a neutral syntactic position.

> By itself, this statement makes sense, but isn't the right "neutral
> syntactic position" for a given fontification region purely a function
> of the region bounds and buffer state?

Well, yes.  And if it's a change triggered fontification, then a function
also of the change data.

> cc-mode should be able to find the right position for any pair of
> buffer positions. If the right start position is before the region to
> be fontified, cc-mode can expand the region to encompass it.

That's precisely what is done.  I'm not sure what you're trying to say
here.

> >> It's the idea that the region "needs to be after brace" that I find
> >> confusing. Shouldn't jit-lock have the right to expand the region
> >> arbitrarily?

> > Absolutely not.  CC Mode needs to do syntactic analysis inside this
> > region, so the region needs to start at the (syntactically) right place.

> Isn't it cc-mode's job to find a good position within or before the
> region font-lock gives it to fontify? cc-mode can nominate a good region
> using the extend-region functions, .... 

CC Mode does indeed find this good position, and communicates it to Font
Lock via the specified interfaces.

> .... but as I see it, there's no contractual requirement that font-lock
> not arbitrarily enlarge this region.

We differ here starkly.  Think about it: the major mode is the absolute
expert on where font-locking should start.  Font Lock itself is merely an
enthusiastic amateur, knowing nothing about the properties of the buffer.
It is impudence of the highest order for Font Lock to override the major
mode.

In practice, Font Lock arbitrarily enlarging the region specified by the
major mode has lead to problems, namely bug #19669, and thence to your
problems with "for".

> > To do this analysis CC Mode must first find the beginning of any
> > definition the current position is in, otherwise context fontification
> > can go wrong.  An example of this going wrong is in this snippet:

> > 1.   template <typename T>
> > 2.
> > 3.
> > 4.   void myfunc(T* p) {}

> > If a space is inserted on L3, this will trigger context fontification of
> > L4.  L4 is fontified differently in the presence of the template
> > declaration (for reasons you, a C++ hacker, probably understand better
> > than me).  What the bug reporter saw was L4's fontification going wrong a
> > half second after typing the space, something as irritating as what is
> > happening to your "for".  The solution I implemented for the problem was
> > to extend the font lock region to include L1.

> Extending the region to L1 is the right behavior. Why can't we perform
> this extension when asked to fontify any range in L1-L4?

We do.

> > That same region extension which includes that template line, in the enum
> > example goes back to after the "{" on L2.  Bug #19669 went wrong in
> > c-font-lock-enum-tail, the function which handles a the start of a
> > fontification region being within an enum brace block.
> > c-font-lock-enum-tail will only trigger when the starting point is within
> > that brace block.  When Font Lock moves the starting point somewhere else
> > (in particular, BOL), that condition no longer holds, and the font locking
> > fails.

> Why can't we check whether any character inside the region to fontify is
> an enum brace end?

What do you mean by "enum brace end"?

-- 
Alan Mackenzie (Nuremberg, Germany).