Re: Syntax ambiguities in narrowed buffers and multiple major modes: a proposed solution.

[Alan Mackenzie]

Hello, Stefan.

On Sun, Feb 26, 2017 at 23:05:44 -0500, Stefan Monnier wrote:
> > I am proposing implementing islands as a solution to the problem, not as
> > a way of thinking about it.

> In order to design a solution, we need to think about the problem somehow.

That's all very well, but we could spend weeks (or months) talking about
the problem without getting anywhere.  With a proposed solution, we can
at least discuss whether it fixes the given problem, and if not, modify it
until it does.  You've already pointed out a sensible enhancement to it.
(Having a third new syntax class which can swap syntax tables without
nesting.)

> >> But the issue is that the syntax beginning in the above example should be
> >> point-min, not 1.

Incidentally, a parse state (like a voltage) is always a _difference_
between two points.  It is determined by (parse-partial-sexp start end
...).  It is an error in thinking to think that there is any a priori
syntax beginning in any buffer situation.

It is only the addition of a syntax cache which creates the notion of
this syntax beginning, but we must never forget that this syntax
beginning is a property of the cache, NOT of the narrowing.  Caches are
complicated things, but narrowing is simple and unambiguously defined.

> > Should it?

Here's that code fragment again, for reference:

    (save-restriction
      (narrow-to-region ...)
      (with-syntax-table ...
        (backward-sexp 1)))


> It's not really inherent in the code, but it's how it currently behaves
> (mostly), and in some cases that is what the author wants.  In other
> cases, the author wants something else.

That code appears to be from .../lisp/obsolete/complete.el, function
PC-lisp-complete-symbol.  If that's the case it's simply using narrowing
without any syntax cache, so there's no concept of a syntax beginning
whatsoever, there.  There would be no need to create an island in that
`narrow-to-region'.

> > When does it actually make a difference, apart from when point-min is
> > inside a string or a comment?  I don't think it makes a difference in
> > the (backward-sexp 1) case above.

> backward-sexp will call back_comment, so yes it makes a difference.
> And since speed may be important, it's also important to make sure we
> don't scan over the 100KB of text that comes before point-min.

:-)  There will be situations where things like backward-sexp will call
back_comment (which is why it is important that back_comment be fast)
but that code fragment isn't one of them.  And even if it did (which
will be rare), it is not doing it inside a tight loop.

> > That is correct.  The creation of an island will not be an expensive
> > action, unless it is in a tight loop - recording two current syntactic
> > elements, and calculating and setting two syntax-table text properties;
> > then restoring these later.

> But won't setting those islands flush the caches (e.g. comment-cache,
> syntax-ppss cache, ...)?

I think you've chosen a bad example for making that point.  The
syntax-ppss cache in the above code will need flushing anyway due to the
with-syntax-table.  The comment-cache cache might or might not need
flushing for that reason (depending on the differences between the two
syntax tables).

Creating an island will invalidate the syntax-ppss cache from the
beginning of the island, yes.  But few of the existing
`narrow-to-region's (there are ~750 in Emacs) will require to create
islands.  This is only needed where the newly created accessible portion
needs to be syntactically separated from what is outside of it.  In most
`narrow-to-region's (like that one in complete.el) this won't be the
case.

The only circumstances this would be concerning would be (i) creating
islands in a tight loop; and (ii) moving forward in the buffer in this
loop.  Are there any such circumstances in Emacs?

I honestly don't believe the flushing of caches is an important problem
in the proposed islands implementation.

>         Stefan

-- 
Alan Mackenzie (Nuremberg, Germany).