Re: A vision for multiple major modes [was: Re: [Emacs-diffs] widen-limits c331b66:]

[Dmitry Gutov]

On 03/27/2016 03:09 PM, Alan Mackenzie wrote:

> I can't really see a proof of concept happening.  Either the thing is
> implemented or it's not.  There's hardly a half way point.

"Full implementation" would work, too. But it would still be considered 
a prototype. Point is, we can't really decide whether we'll end up using 
it, before trying it for a while.

> > > That will require more investigation.  But syntax based searching and
> > > regexp based searching will certainly be amongst them.

I don't think that's enough. E.g. (goto-char (match-end n)) is a common 
idiom for syntax highlighting and indentation code.

> > [island-affected primitives]
> looking-at? Probably.
> char-after? No.
> syntax-after? No. (This works only on a single char, hence must be in a
>               single island, it can't span two.)
> forward-char? Maybe.  In fact, there's a good argument for not moving
>               forward when `forward-char' hits an island boundary.

Being confined to the current island is not what I had in mind. If 
that's the idea of how such primitives would work, I don't really see a 
lot of benefit over using narrowing around all those calls.

> > Speaking of the last one, will the buffer positions, as visible to the
> > submode code, be renumerated, or will they have gaps?
>
> It would retain the global buffer position, for consistency with the rest
> of Emacs.  For example, in narrowed buffers, the position relative to
> point-min is never used.

True. I'm more worried about gaps. But they could be treated like 
whitespace, I suppose.

> > Depending on your answer to the previous question, even simply inserting
> > text inside one of the preceding islands might make syntax-ppss-cache
> > out of date in the current island.
>
> That could probably be solved by making positions in that cache relative
> to the beginning of the island, and things like that.  I think you'd want
> to make `syntax-propertize--done' island local, too.

Right. So then, some yet-unknown body of code has to become 
island-aware, and the improvement is not that seamless anymore.

> > > > Next, at which points exactly would Emacs look at the island boundaries
> > > > and change the island-local variables to the values set in the current
> > > > island? Probably not after each point movement. In post-command-hook?
> > > > That's also already done.
>
> > > It wouldn't use any high level facility like post-command-hook.  The
> > > mechanism would be more like a buffer local variable, entirely handled
> > > by the C level, so that such a binding would simply exist.
>
> > You didn't answer my question, though. When will the bindings apply?
>
> Sorry, an island local binding would be current when point is within that
> island.  Or, perhaps there could be some variable which would be set to a
> position to do this job.

What if var has an island-local binding? And my function has this:

(setq var 1)
(goto-char xx) ; where xx is in a different island

will its value change mid-program? What if xx is in the same island? 
Will the value change then?

I don't think either should be true. Then, the "adequate" model would 
amount to changing them in post-command-hook anyway.

> > I don't know if we'd need the island-beginnig/island-end accessors in
> > your proposal.
>
> I think we would.  (narrow-to-region (point-min) (point-max)) is a null
> operation.  If the major mode has narrowed, and needs to set point-min to
> "1", it will need to know the island-beginning, somehow.

What if it just calls widen?

> > > If you "whitespace" a region out, you've got to remember
> > > the text properties that were there originally, and restore them
> > > afterwards.  There's no easy way to do that.
>
> > I think it's doable using char-property-alias-alist. The multi-mode will
> > define its own property as an alias of `syntax-table', and then put it
> > on and remove it from text at will.
>
> I don't think that would work at the moment.  syntax-table text
> properties would take priority over syntax-table-1 properties.  Something
> similar to char-property-alias-alist, but giving syntax-table-1 priority,
> would need to be implemented.

You're probably right.

> It would relieve super modes of the tedious necessity to maintain island
> local variables using functions in post-command-hook, and things like
> that.

They'd still have to maintain the island boundaries. And the lists of 
variables to set island-locally, or mode-locally, or chunk-locally. That 
constitutes the majority of the related code already.

> It would not be necessary to use widening and narrowing to
> restrict indentation/fontification code to an island.

So making it seem, to the Lisp code, like the multiple related islands 
don't have anything (for some definition of "anything") between them is 
not a priority? That would be the high point of the "islands" idea from 
my perspective. But having only some primitives affected would likely 
break this use case.

Here's an example of ERB code Vitalie brought up recently:

<% if foo %>
  <%= bar(boo) %>
<% end %>

The parts of the buffer between %'s would be Ruby islands. The 
suggestion was to use the value returned by the indentation function in 
the second island (second line) to indent the "real" buffer contents.

But: neither of the islands contains a newline. If they are combined in 
the most straightforward way, the Ruby line will be 'if foo bar(boo)', 
and its indentation would be not what we're looking for. I think the 
current ways to look for a newline,

  (progn (skip-syntax-backward " ")
         (eq (char-before ?\n))

or

  (looking-at "\\s-$")

will fail to work. Also note that neither of the two islands in this 
example contains the newline in question.

To sum up, hard examples are hard. And yet, the new feature should make 
some use cases possible that were impossible before, not just allow for 
easier island-local variables.

> I don't think having islands start at position 1 is a good idea.  But
> having the relevant primitives skip the gaps between chained islands, and
> stop at an island boundary when not chained, when `restrict-to-island' is
> non-nil, would be a central idea.

The "not chained" idea would require some explanation.