Re: [Emacs-diffs] scratch/widen-less a4ba846: Replace prog-widen with consolidating widen calls

[Dmitry Gutov]

On 12/4/17 5:52 PM, Eli Zaretskii wrote:

> Your work on this is highly appreciated.

Thank you.

> But I have certain
> responsibilities, and I'm trying to do my job here.

True, and I fully expect to discuss API breakage, development cycles, 
and that sort of issues. Adding a big requirement, and it being 
something we've effectively given up on solving in the shorter term, has 
been very surprising.

> I'm also your fellow developer of the same project and its active
> user.  I think it means my opinions should be of some importance, not
> something to be ignored solely because I'm not coding this.

I kind of expected opinions in different directions.

> Just like
> I would never dream of ignoring your opinions about features I'm
> implementing.

Not opinions like "don't push this change, unless you make it work for 
my use case X too!", I expect. At least, not for all such opinions.

> > [C-like major modes] can be catered to, *as soon as* the authors of each 
> > individual major mode make the effort to support the feature.
>
> Agreed.  What I'm trying to establish is whether the design supports
> those major modes, or does it ignore them.  The latter I would like to
> avoid as much as possible.

Whatever special requirements particular major modes turn out to have, 
can be added later. I've tried my best to make sure that improvements 
would be smooth. Hopefully just additive.

> > Eli, we already have a feature in Emacs (prog-indentation-context) that does 
> > not adhere to your (unreasonably high) requirements.
>
> Emacs development is not only about not losing existing features.  It
> is mainly about gaining new important features.  Here you are adding a
> feature that I think is important enough for us to try make it support
> C-like languages.

So far, the discussion of supporting it in CC Mode, has not uncovered 
any extra requirements to the proposed protocol, from where I'm standing.

> I don't think this requirement is unreasonably
> high, as those languages are still pretty much the mainstay of the
> industry, and definitely used by many Emacs users.

The requirement seems a bit late, is all. Either you apply it to 
prog-indentation-context and vote to remove it, or you don't get to 
apply it to this proposal. That's my opinion, at least.

> > And that cache point to a position outside of the current restriction. 
> > Confusion ensues. I've seen errors originating from that (I think), but can't 
> > recall the exact sequence of calls.
>
> Thanks, so the caches will have to be sensitive to restrictions as
> they are sensitive to deletion of text.

Right, and that's the responsibility of the major mode. So there's not 
much we can add in terms of code (I think), but we can add some more 
sentences to the manual.

> But given your example with CC Mode's caches, that's not all of it, is
> it?  You also need the mode to adapt any such caches to changes in
> buffer restrictions, right?

I think so. But only CC Mode, in my experience, includes such caches 
that lead to problems in MMM context.

> > We keep 'prog-first-column' from that proposal
>
> But it was a function, and you made it a variable.  This will get in
> the way of compatibility, and also potentially will make future
> extensions harder.  Why not keep it a function?

To have a better API. If we keep it a function, do we also have a 
variable prog-first-column?

Then, some consumers might have doubt which ones to use, and might opt 
to reference the variable. Then, we lose all benefits of having it a 
function (like making its implementation somehow smarter later), and the 
only benefit remaining is backward compatibility of having a function 
with that name.

But! Like with PREV-CHUNKS, prog-first-column (and the later prog-widen) 
are supposed to be used by the major modes only. There are no references 
to either of these functions in the latest antlr-mode.el, and there 
shouldn't be.

And as long as all the calls to that function are inside Emacs, we're 
free to change it however, including turning it into a variable.

> > and instead of allowing MMM to indicate the chunk bounds through 
> > prog-indentation-context, we allow MMM to apply narrowing directly, and that 
> > modes honor it.
>
> This simplification also took away some of the features, like the
> ability to nest restrictions.  I wonder why did you discard that.
> It's existing functionality, pretty lightweight, which was in Emacs
> for some time, and is reasonably well documented.  And it already
> satisfied your requirement of not allowing sub-modes to widen.  Why
> not leave it alone?  What am I missing?

It's just pointless. I've asked, in the original discussion 2 years ago, 
for at least one patch that would use it. None have arrived since.

And as long as major modes do not use it, MMM packages don't have any 
benefits from it.

> >          Transitioning from prog-indentation-context to the new approach is very
> >          easy.
> >
> >      That's what I thought.  And that's why we should do that
> >      simultaneously with landing the new approach.  There's no reason to do
> >      that earlier.
> >
> > Why confuse the language modes authors?
>
> If they should be confused (I don't think so), they are already
> confused, as we've been having this for the last 2 years.

Not in a release version. And the confusion I'm talking about will come 
later, when we either try to keep both ways of doing this (I don't think 
we'll manage), or abruptly switch from one way to another.

The latter is easy code-wise, but by that time, whatever compatibility 
code they had, they might have removed it intending to support only 
Emacs 26+, for instance. And here we come saying Emacs 27 will be 
strictly incompatible.

> >      We cannot know that.  Once the code is that long with us, we cannot
> >      throw it away without a equivalent replacement.  And I see no reason
> >      to do that now, since the fact that this code will exist in Emacs 26
> >      doesn't hurt anyone, especially since the replacement will be easy, as
> >      you say.
> >
> > Well... you have a point there. The result will be pretty ugly, though. Adding 
> > an API in one version, and removing it in another.
>
> We will have to discuss the "removal" part.  An alternative is to
> provide compatibility shims, or even leave (some or all of) the
> existing API intact.

Yeah, I cannot imagine how a compatibility shim would work. Hence my 
request to replace it as soon as possible.

> > They are incompatible with each other, so it's not like there can be a smooth 
> > transition.
>
> I don't see the incompatibilities.  Basically, you replaced prog-widen
> with widen,

Nope, just removed prog-widen. And widen in one place.

Also replaced one prog-widen call with widen, but it was not in an 
indentation function (os it probably shouldn't have been there in the 
first place).

> made prog-first-column a variable instead of a function,

See (*).

> and tossed prog-indentation-context.  We could instead keep the
> existing API with minimal changes: prog-widen calls widen internally,
> and we could allow direct calls to widen as well.

That won't work. Either indentation functions call prog-widen (which 
fully widens in the absence of prog-indentation-context binding), or 
they don't widen at all. These are two different, incompatible ways of 
doing things.

> IOW, the transition could be much smoother than it will be if we
> actually remove that stuff, because I don't think there's any
> incompatibility here which would disallow direct calls to 'widen'
> and/or leaving prog-indentation-context at its default nil value.

Yes, there is. Please go back to one of the several descriptions, in 
this thread only, of how MMM uses narrowing to restrict indentation 
engine to the current chunk.

> Or
> maybe I'm missing something again.  But if I'm right, and if we can
> make the necessary changes limited only to the documentation, then it
> could well make it into Emacs 26.  So I think it is worth our while
> to make some effort in that direction, if at all possible.

Unfortunately, I don't imagine it's really possible.