Re: A vision for multiple major modes: some design notes

[Eli Zaretskii]

> Date: Wed, 20 Apr 2016 19:44:50 +0000
> From: Alan Mackenzie <address@hidden>
> 
> This post describes my notion of how multiple major modes {c,sh}ould be
> implemented.  Key notions are "islands", "island chains", and "chain
> local" variable bindings.

Thank you for publishing this.  A few comments and questions below.
Please keep in mind that I never had to write any Lisp that deals with
these issues, so apologies in advance for possibly silly questions and
misunderstandings.

>   o - To the user, the current major mode will be that of the island where
>     point is.  All familiar commands will work without restriction.

Does this mean the display of mode line, menu bar, and tool bar will
change accordingly?

A more subtle issue is with point movements that are not shown to the
user (those done by Lisp code of some command, before redisplay kicks
in) -- what will be the effect of those? do they trigger redisplay,
for example?

>   o - An island chain will have @dfn{chain local} variable bindings.  Such a
>     binding will become current and accessible when point is within one of the
>     chain's islands.  When point is not in an island, the buffer local binding
>     of the variable will be current.

Emacs sometimes examines buffer text without moving point, and we
generally expect for buffer-local bindings to be in effect regardless.
A prominent example is the display engine.  I will return to that
later.


>     * - [Island] will be covered by the text property `island', whose value 
> will be
>       the pertinent island or island chain (see section (ii)) (not yet
>       decided).  Note that if islands are enclosed inside other islands, the
>       value is the innermost island.  There is the possibility of using an
>       interval tree independent of the one for text properties to increase
>       performance.

I don't understand the notion of "enclosed" islands: wouldn't such
"enclosing" simply break the "outer" island into two separate islands?

>   o - `scan-lists', `scan-sexps', etc. will treat a "foreign" island as
>     whitespace, much as they do comments.  They will also treat as whitespace
>     the gap between two islands in a chain.

Why whitespace? why not some new category?  By overloading whitespace,
you make things harder on the underlying infrastructure, like regexp
search and matching.

>   o - The regexp engine will be enhanced such that the regexps "\\s-", "\\s ",
>     and "[[:space:]] will match an entire island.

Extending [:space:] that way seems to be an implementation detail
leaking to user level.  I think we should avoid that at all costs.

>   o - The gap between two islands in a chain will also be matched by the above
>     regexps.
>   o - This treatment of an island, and a gap between two islands, as WS will
>     occur only when `in-islands' is non-nil.
>   o - When `in-islands' is nil, there will be no reliable way of scanning over
>     an island by regexps, since it is a potentially nested structure, and FSMs
>     don't recognise arbitrarily nested structures.

> (vii) Variables.
>   o - Island chain local variable bindings will come into existence.  These
>     bindings depend on the island point is in.  There will be lower level
>     routines that will have "position" parameters as an alternative to using
>     point.
>   o - All variables which are currently buffer local will become chain local
>     except for those whose symbols are given a non-nil `entire-buffer'
>     property.  There will be no new functions like
>     `make-chain-local-variable'.
>   o - When the `entire-buffer' property is nil, the buffer local binding of a
>     variable will hold the value pertinent to the areas of the buffer outside
>     of islands.  When that property is non-nil, the binding holds the value
>     for the entire buffer.
>   o - When `in-islands' is nil, the chain local mechanism described here is
>     not used - instead the familiar buffer local binding is used.
>   o - The current binding for a local variable will be the chain local binding
>     of the island chain of the island containing point.  If point is not in an
>     island, the buffer local binding is current.
>   o - If a chain local binding is current, and its value is unbound, the
>     binding of an enclosing scope is NOT used in its place.  Probably the
>     variable's default-value should be used when reading.
>   o - In buffer.h, a new macro CVAR ("island chain variable") analogous to
>     BVAR will be introduced.  It will use BVAR as a fall back.  Most
>     invocations of BVAR will be changed to CVAR.
>   o - In data.c, the mechanism for accessing local variable bindings
>     (e.g. `swap_in_symval_forwarding') will be enhanced to test `in-islands'
>     and handle chain local bindings appropriately.

I'm not sure I understand the details.  E.g., where will the
island-chain local values be stored?  To remind you, buffer-local
variables have a special object in their symbol value cell, and BVAR
only works for the few buffer-local variables that are stored in the
buffer object itself.  I'm not sure I understand how CVAR could solve
the problem you need to solve, which is keeping multiple chains per
buffer, each one with its values of these variables.

> (ix) Miscellaneous commands and functions.
>   o - `point-min' and `point-max' will, when `in-islands' is non-nil, return
>     the max/min point in the visible region in the same chain of islands as
>     point.
>   o - `search-\(forward\|backward\)\(-regexp\)?' will restrict themselves to
>     the current island chain when `in-islands' is non-nil.
>   o - `skip-\(chars\|syntax\)-\(forward\|backward\)' will likewise operate in
>     the current island chain (how?) when `in-islands' is non-nil.
>   o - `\(next\|previous\)-\(single\|char\)-property-change', etc., will do the
>     Right Thing in island chains when `in-islands' is non-nil.
>   o - New functions `island-min', `island-max', `island-chain-min' and
>     `island-chain-max' will do what their names say.
>   o - There will be no restrictions on the use of widening/narrowing, as have
>     been proposed for other support engines for multiple major modes.
>   o - New commands like `beginning-of-island', `narrow-to-island', etc. will
>     be wanted.  More difficultly, bindings for them will be needed.
>   o - ??? Other commands to be amended.

This actually sounds like a simple extension of narrowing, so I wonder
why do we need so many new object types and notions.

> (x) Emacs subsystems and `in-islands'.
>   o - Redisplay will bind `in-islands' to non-nil, but will successfully
>     display all islands wholly or partially in windows being displayed.
>   o - Font Lock will bind `in-islands' to non-nil, but will successfully
>     fontify all pertinent islands.
>   o - `island-before/after-change-function' will be called with `in-islands'
>     nil.
>   o - `before/after-change-functions' will be called with `in-islands' bound
>     to non-nil.
>   o - Major modes will need to bind `in-islands' to non-nil for such things as
>     indentation.
>   o - For normal user interaction, `in-islands' will be nil.

I don't see any discussion of how redisplay will deal with islands.
To remind you, redisplay moves through portions of the buffer, without
moving point, and access buffer-local variables for its job.  You need
to augment the design with something that will allow redisplay see the
correct values of variables depending on the buffer position it is at.
The same problem exists for any features that use display simulation
for making decisions about movement and layout, e.g. vertical-motion.

More generally, perhaps it will help if you publish the rationale for
at least the main points of this design, discussing possible
alternatives and explaining why you ended up with the one you present
as the design decision.  This could help us see the main issues that
are to be dealt with, and perhaps suggest better ways of dealing with
them.  Seeing just the final product of the design tends to limit the
discussions to low-level details, which could easily miss the broader
picture and issues.

Thanks.