Re: [MMM] Re: narrow-to-here-document

[Michael A. Shulman]

On Thu, Jul 03, 2003 at 08:32:43PM -0400, Stefan Monnier wrote:
> I think we need feedback from the mmm-mode people before making any
> decision on such an issue.  I haven't looked at mmm-mode recently, 
> but it used to try and deal with font-lock, indentation and stuff like
> that.  I'd actually expect them to need/want ad-hoc support from
> font-lock.el (for example) but haven't heard anything (maybe they
> complain about such things in their own .el files rather than here).

Our current implementation is able to do reasonably well by using
font-lock-fontify-region-function and font-lock-beginning-of-syntax-function,
but we'd love to have better support from font-lock (and other
packages) for this sort of thing.  Thanks, Stefan, for crossposting to
the mmm list, since I wasn't following emacs-devel.

Briefly, the way mmm-mode works is to parse the buffer for regions
that it thinks (based on its configuration) ought to be edited using a
different major mode, and create overlays to mark those positions.
(It's not currently as good as we'd like at updating these overlays
dynamically as text is edited, but I believe this is just because no
one has sat down to write good code to do it.)  Then it uses
post-command-hook to check which region the point is in and make the
buffer look (almost) like it's in the associated major mode.  It
primarily does this by changing keymaps, syntax tables, and lots of
other buffer-local variables.  Some of these values are stored
buffer-locally, and others region-locally (all managed manually, of
course).

I don't mean to co-opt this disucssion for mmm, since it started on a
somewhat different (though related) topic.  But a "wish list" of
possible features in emacs to make mmm and related packages easier has
been on my mind for a while, and the discussion has been moving in
that direction.  So here are a few things I think emacs could do to
make this sort of stuff easier and more effective.

1. Have a "registry" of some sort for major modes, where they can
store enough information for mmm to turn them on effectively in small
regions.  Currently, the way mmm finds out what local variables and so
on to set for each mode is to create temporary buffers and save the
values of individual local variables.  Among other things, this
requires mmm mode knowing about all local variables that any major
mode could want to set, and so we have a huge list of what variables
to look for and what modes to look for them in.  In addition, some
major modes expect things like a file name set in all buffers, so we
have to do kludgy things to the temporary buffer to try to make them
work.

2. Have some sort of built-in support for variables that are localized
to a text region within a buffer, such as an overlay or a text
property.  I'm not sure how or if this would work in general, but it's
effectively what mmm does already to some extent.  In particular, some
variables want to keep different values in different regions even of
the same major mode, in particular variables that store the state of
syntax parsing.  This leads to...

3. Better support in the syntax parsing routines allowing the caller
to specify starting and ending positions and states.  One of
our biggest problems is that submodes end up trying to parse their
syntax beyond the beginning of their region and getting confused, so
ideally we would be able to tell them where to start their parsing,
and what state to start parsing in.  Sometimes the start state needs
to be a continuation of the end state of the previous region in that
mode (as with some literate programming, PHP, etc.), but sometimes
it ought to be an independent start for each region.

This is similar to the 'syntactic realm' idea that David Kastrup
suggested.  I don't think we've tried using intangible properties, for
the reasons mentioned and perhaps others, but Joe Kelsey has had some
success using syntax-table text properties to make the text that's not
supposed to be parsed be treated as a single word or as whitespace, as
appropriate.  Joe, want to comment?

4. Effectively the same thing for font-lock, which uses the syntax
parsing functions but also its own parsing functions.  In general
we've had more success hacking around font-lock than around syntax,
because font-lock provides more customization variables and its state
is easier to understand.

If we want to continue this discussion, should it stay crossposted, or
only on emacs-devel?

-- 
Michael A. Shulman
http://www.ugcs.net/~shulman

The goal of psychological, as of biological, development is
self-realization, or individuation.  But since man knows himself only
as an ego, and the self, as a totality, is indescribable and
indistinguishable from a God-image, self-realization... amounts to
God's incarnation.
        -- Carl Gustav Jung