bug#36136: [PATCH]: Re: bug#36136: syntax-ppss fails to invalidate its cache on changes to syntax-table text properties

[Alan Mackenzie]

Hello, Stefan.

On Wed, Jun 12, 2019 at 04:37:25 -0400, Stefan Monnier wrote:
> > The following patch is simpler than my first proposal, following
> > feedback from Eli.  It works for me.

> > Stefan, could you look at this, please?

> Here I am.

Hello!

> > * src/textprop.c (syntax-propertize--done): New buffer local variable.
> > (set_properties, add_properties, remove_properties): when a syntax-table 
> > text
> > property is being changed, reduce syntax-propertize--done to the buffer
> > position.

> Hmm... I'm not too fond of adding ad-hoc support for specific
> text-properties in (set_properties, add_properties, remove_properties).

Neither am I, particularly.  But the whole point of syntax-ppss, surely,
is that it should work automatically, without users having to call
syntax-ppss-flush-cache all the time.

The root of the problem is that inhibit-modification-hooks is too blunt
a tool.  Setting it prevents running functions we want to run, just as
much as ones we don't.

I've just had another idea: we introduce a new property called something
like dont-inhibit.   When a function in before/after-change-functions
has this property, it would run, regardless of
inhibit-modification-hooks.  Or possibly, we could introduce new hooks
no-inhibit-before/after-change-functions.

What do you think?

> > * lisp/emacs-lisp/syntax.el (syntax-ppss--trim-cache): New function 
> > extracted
> > from syntax-ppss-flush-cache.
> > (syntax-ppss-flush-cache): Now only modifies syntax-propertize--done and
> > syntax-ppss--done.
> > (syntax-ppss): Calls syntax-ppss--trim-cache and sets 
> > syntax-propertize--done.

> This part looks OK.

> I'm not sure if making the cache-flushing more lazy will be a win
> overall: it speeds up buffer modifications at the cost of slowing down
> syntax-ppss.

It will probably not make a great deal of difference either way.  Buffer
changes are frequent in Emacs, and so are calls to syntax-ppss in many
major modes.

> To get back to the original problem:

> > This is because the invalidation function, syntax-ppss-flush-cache is
> > invoked only as a before-change function, but typical (?all)
> > syntax-table property changes happen when before-change-functions
> > is inactive.

> That's why it doesn't have "--" in its name: if you don't want to use
> syntax-propertize then you'll probably have to call that function
> by hand.  I consider it as perfectly acceptable.

Well, for CC Mode I'm going to have to do that anyway, since Emacs-26.x
and earlier are already out there and aren't going to change.  This is
going to be tedious and error prone.

But for the future, it would be nice if syntax-ppss could take note of
all buffer changes, rather than just some of them.

>         Stefan

-- 
Alan Mackenzie (Nuremberg, Germany).