Re: Bug #25608 and the comment-cache branch

[Alan Mackenzie]

Hello, Stefan.

On Sun, Feb 12, 2017 at 12:49:29 -0500, Stefan Monnier wrote:
> > I also have an attachment to it because it works, and would save me
>                                                  ^^^
>                                                for you

> > demoralizing work debugging bugs caused by open parens in column zero in
> > comments.

> Is that really the only reason?  It seems like an amazingly complicated
> way to go about it.

comment-cache is a gross simplification of syntax.c, scanning comments
only in the forward direction and totally eliminating
open-paren-in-column-0-is-defun-start from syntax.c.  To achieve this
simplification took a lot of work.

> Let's see some alternatives:
> - set open-paren-in-column-0-is-defun-start to nil.

Too slow.

> - add a font-lock rule which highlights open-paren-in-column-0 inside
>   comments in bright red.

Totally irrelevant to what's required.  By the way, did you ever get
around to looking at that bug which noted that mechanism no longer
working?

> - use my syntax-ppss-based patch.

Doesn't work all the time, in particular in narrowed buffers.

> > Bug #22983 is a flaw.

> Great!  We're trying to have a reasoned argument; I tell you that using
> this term to describe this problem is not helping and you insist on
> using it.  From where I stand, this qualifies as provocation.

Of all the words in English which mean "imperfection", that's one of the
milder ones.  Would you prefer I used "defect" or "fault" or something
else?  #22983 _is_ a flaw/defect/fault/whatever else you want to call
it.  In particular, syntax-ppss gives out different results for a buffer
position depending on its internal state.

If that word would provoke you into actually fixing #22983, after all
this time, I would use it again for that purpose.

> > Also the cache invalidation in syntax-ppss is less than rigorous.

> Yup, syntax-ppss's implementation is not perfect.  That can be improved.

> > For example, the cache isn't invalidated when syntax-table text
> > properties are applied or removed.

> This is not a correct characterization of the most common
> cache-invalidation problem with syntax-ppss: there's a correlation
> between the problem and syntax-table text properties, but that's all: it
> also affects all other properties, but it doesn't affect all changes to
> the syntax-table text properties.

syntax-ppss is a cache of the syntax-table text property.  Not
invalidating the cache when a syntax-table text property is changed is
an imperfection.  Will it be fixed?

By contrast, there are no known bugs in the cacheing in comment-cache.

> >> BTW, your comment-cache doesn't fix that "flaw" and hence won't help any
> >> of those users of syntax-ppss which can't be changed to use your
> >> comment-cache.
> > That's incoherent.  comment-cache was never intended to help those other
> > uses, though it appears it could do so for most of them.

> It's only incoherent if you refuse to see the larger picture.

The larger picture is that comment-cache can work alongside syntax-ppss
pefectly happily without any contention.

> > Can't be done, as I keep telling you.  comment-cache is solely for
> > handling literals.

> Then it's useless, AFAIC:
> - we need syntax-ppss's data for lots of things.
> - your code can't replace all those uses.
> - so we're stuck with syntax-ppss, no matter how much you think it sucks.
> - then we might as well use it in back_comment instead of your code,
>   since it's there and is cheap.

But it doesn't work properly.  comment-cache is also cheap, having
already been written and debugged.

> > The question of "widening" is not difficult.  Narrowing a buffer should
> > not change the syntax of the characters in it.  Doing so leads to
> > inconsistencies.

> I can agree with that.  But currently that's not how Emacs behaves, so
> it's an incompatible change (which I'm quite willing to make, BTW), and
> needs to come with some way to recover the other behavior when that one
> is needed.

> > If I understand correctly, the problem is that multiple-major-mode modes
> > are trying to use narrowing to get a null syntactic context.

> That's the typical example, but not the only one.

> > They are trying this because we don't provide anything better.
> > We should provide something better.

> Agreed.

> > I suggested such a something last spring ("islands").  If each buffer
> > position has an unambiguous syntactic context the question of
> > "widening" simply evaporates.

> I think this is too specialized to a particular application (multiple
> major modes).  We also need to accommodate other cases.

Could you identify these other cases?

> For that we need to provide something equivalent to

>     (save-restriction
>       (narrow-to-region BEG END)
>       ...)

> but where syntax-ppss and friends will know that we shouldn't widen past
> BEG/END ....

I thorougly dislike the conceptualization of handling syntax as
"widening".  Both the Lisp and C parts of Emacs use narrowing and
widening "all the time", and if we try to express semantics in terms of
"widening" and "narrowing" we're going to create confusion.

> .... and that BEG should be taken as "the (temporary) beginning of the
> buffer".  Let's call it `with-region-as-subbuffer`.  Most likely, this
> new functionality should also make it possible to temporarily provide
> a different syntax-table.  Such things are used in various
> circumstances where the author simply wants to reuse Emacs's syntax.c
> code to avoid writing some ad-hoc parsing.

> IOW, we need to provide something on top of which we can build this
> `with-region-as-subbuffer` macro as well as your islands.

Introducing the new syntactic symbols "island start/end" would cater for
with-region-as-subbuffer admirably, without having to resort to
confusing narrowing.  Every buffer position would continue to have its
unique (global) syntactic context.

>         Stefan

-- 
Alan Mackenzie (Nuremberg, Germany).