Re: Further CC-mode changes

[Alan Mackenzie]

Hello, Stefan.

Sorry I was a bit vague about one or two things last night.  I'll try
and clear them up.

On Fri, Sep 12, 2014 at 11:04:26PM -0400, Stefan Monnier wrote:
> >> - BTW, I think it would be better not to enable abbrev-mode (let it be
> >> a user's choice).  IIUC CC-mode relies on abbrev-mode internally
> >> for some of its electric-indent functionality, so we'd have to use
> >> some other approach (e.g. a post-self-insert-hook).
> > I tend to agree, having been a bit disgusted myself when I discovered how
> > "else" and friends were being electrically indented.
> > after-change-functions might be better here, since it's more general, and
> > also better supported.

> I think "more general" is a disadvantage in this case (I'd rather not
> have auto-indentation after I paste a space).
> As for better supported, I don't care too much: I'd just keep the old
> abbrev code for the Emacsen that don't have post-self-insert-hook yet.

Yes.  That's a bit messy, though.  But I think one could legitimately
categorise the mandatory enablement of abbreviation mode as a bug.

[ ... ]

> >> - Move awk-mode's actual definition to cc-awk so as to get rid of the
> >> (require 'cc-awk) and the ":syntax-table nil" hack.
> > Would leave traps for maintainers,

> Such as?

A change to all of c-mode, c++-mode, ....., pike-mode.  There's a danger
that the maintainer will forget awk-mode, since it's not there any more.

> > and break uniformity between the modes.

> Actually, I think the right move would be to move all awk-related code
> into cc-awk.el, thus making it more uniform with the various externally
> maintained cc-modes.

Another simplification would be simply to load cc-awk.elc along with the
rest of CC Mode, on the grounds that the 19k file is no longer big enough
to worry about on modern machines with gigabytes of RAM.

> > This change wouldn't enhance functionality or maintainability.

> The code is cleaner and simpler with the change than without.

Yes, maybe, but....

> >> - make awk-mode use syntax-propertize when available.
> > Why?

> You know why: `syntax-propertize' is used by all major modes except
> cc-mode derived ones.  And it's more efficient, because it's more lazy.

Lazy?  Does C-M-f, for example, trigger syntax-propertize in any way?

> Also its correctness argument is much simpler since it doesn't rely on
> interactions between before-change-functions, after-change-functions
> (and font-lock), contrary to your code.

The argument could run that because syntax-propertize doesn't take
account of these subleties, its actions aren't always correct.  But ...

syntax-propertize isn't documented in the Elisp manual.  It's unclear
when it gets called (in relation to after-change-functions, for example)
and even _if_ it always gets called (e.g. when font-lock isn't enabled).
Adapting code to use it is thus a _massive_ amount of work - it involves
reading the source code for syntax-propertize and friends to work out
what it does.  I suspect that for a mode which needs syntax-table text
properties early in its after-change-functions, syntax-properties
wouldn't work.  I don't think CC Mode comes into that category (though
I'm not sure).

> > It's a lot of work to change,

> It's work I did, pro bono.  Just enjoy it.

No, you haven't.  What's the equivalent of `syntax-propertize' in XEmacs,
or an old GNU Emacs?  "Somebody" (tm) has to write a compatibility macro
which will look something like this:

(defmacro c-set-syntax-propertize (f)
  (if (boundp 'syntax-propertize-function)
      `(setq syntax-propertize-function ,f)
    `(????
    )))

What exactly goes in "????"?  So many changes you want to make are of
this nature - using new facilities of Emacs to replace something which
works fine anyhow, and then leaving somebody else, me, to take care of
backwards and XEmacs compatibility.

> > syntax-propertize is only available in relatively recent versions of
> > GNU Emacs, and the documentation suggests that syntax-propertize is
> > not called after buffer changes in general, the documentation not
> > being entirely clear on this point.

What I meant here was it is not clear when (or even whether)
syntax-propertize gets called when font-lock is not enabled.  If s-p is
not called with disabled font-lock, I think you should state this
explicitly somewhere in the Elisp manual and/or doc string (or fix the
problem and then document the fixed function).

> Indeed it's not.  If you want to be sure that syntax properties have
> been applied upto POS, you need to call (syntax-propertize POS).

Right.  So I'm now going to have three alternative code flows in place of
the one I currently have: (i) One flow for when font lock's enabled; (ii)
one for when FL is disabled, which explicitly calls syntax-propertize
from after-change-functions; (iii) one for Emacsen lacking
syntax-propertize.  I've already got (iii).  I think you're asking me to
add (i) and (ii).  This is an increase in complexity and a lot of work.

> > This change wouldn't enhance functionality or maintainability.

> Yes, it enhances maintainability of awk-mode by making it work more like
> all other modes.  It also stops cc-mode from doing all kinds of weird
> unrelated crap having to do with maintaining c-found-types and
> c-state-cache, which are not used in cc-awk.

c-state-cache IS used in AWK; it's used in all CC Mode modes and is
fundamental to parsing braces etc.  c-found-types isn't, since there are
no types in AWK.  Probably the neatest solution here would be to move the
c-found-types stuff into its own function, called via the
`c-get-state-before-change-functions' mechanism, or similar.

> >> - Use define-derived-mode's built-in keymap inheritance rather than
> >> c-make-inherited-keymap.
> > c-make-inherited-keymap is simpler.

> Huh?  Have you even looked at the patch?  There is nothing to do to use
> define-derived-mode's built-in keymap inheritance, so how can your extra
> code be simpler?

OK, maybe it's not that bad.  define-derived-mode does exist in (at
least) newer versions of XEmacs.  But it seems to mean having at least
one "artificial" mode which will add to complexity.  define-derived-mode
is a complicated macro, with so many implicit things, that often the only
way to see what it does is with `macroexpand'.

> > This change wouldn't enhance functionality or maintainability.

> It removes unneeded code.  That improves maintainability.

> >> - initialize c-mode-base-map in its defvar.
> > An inconsequential change that wouldn't enhance functionality or
> > maintainability.  It's work to make, though.

> It just makes code more standard, with no known downside.

But with no upside either.  The downside, as already said, is the work
involved in doing it and propagating the change to stand alone CC Mode.

> >> - fix c-after-font-lock-init so it doesn't accidentally add
> >> a c-after-change which wasn't there to begin with.
> > ???  There's nothing broken here, surely?

> The current code of c-after-font-lock-init has a bug, but no the current
> cc-mode does not trigger this bug.

> The bug is that c-after-font-lock-init is supposed to *move*
> c-after-change to the front of after-change-functions, but in case
> c-after-change is not in after-change-functions, it ends up *adding*
> it instead.

In CC Mode, c-after-change IS on after-change-functions.  This is
fundamental.  There is no situation in which this can't be the case.

> Your approach to syntax-table properties requires you to be extra
> careful about ordering between c-after-change and font-lock's own
> after-change-functions, hence the c-after-font-lock-inithack.  But if
> for some reason some cc-mode (such as cc-awk when it uses
> syntax-propertize) does not need/want c-after-change, the current code
> in c-after-font-lock-init will end up making it needlessly difficult to
> remove c-after-change from after-change-functions.

See above.  I really can't foresee any circumstances where a CC Mode mode
won't need before and after change functions.  Both c-before-change and
c-after-change are equipped with a mechanism to have mode dependent
functions called.  Maybe a little more modularisation using these
facilities wouldn't go amiss.

> >> - Run c-update-modeline from after-change-major-mode-hook and
> >> hack-local-variables-hook rather than from the major mode function.
> >> This lets us fix the bug where c-mode-hook is run twice and saves us
> >> from having to define c-run-mode-hooks.
> > after-change-major-mode-hook is a GNU Emacs speciality.

> That's why I also do it from hack-local-variables-hook.

> > How about enhancing define-derived-mode so that modes that have things
> > to do after running their hooks can do them?

> That's not late enough, since you need to do it after setting the
> file-local variables.

How about doing it anyway?  A major mode incorporating user changes made
in its mode hook into its current state is a reasonable thing to expect
to do.

> In this particular case, the better fix would be to introduce a new
> `mode-name-sidekick' in the mode-line-format, so we can get rid of
> c-update-modeline altogether.

I don't think XEmacs has the capability of running functions from mode
line format elements.  I looked, once.

> >> - make c-mode-common into an actual parent mode.  There is a naming problem
> >> since we have c-mode-base-map for the "parent map" but
> >> c-mode-common-hook for the "parent hook", and also because this "mode"
> >> doesn't end in "-mode", but I think we can live with it.
> > Why?

> Why not?

>From a user point of view there is no "mode" here - artificially
introducing it would merely create confusion.

> > There is no such mode c-mode-common; nobody's ever going to have a
> > buffer in this mode; it would merely be a software-engineeringy way of
> > achieving... what?  It would add complexity where none is warranted.

> What complexity?  It reduces code duplication, if anything.

The complexity is in having a deep inheritance tree of modes and "modes".
The number of duplicated code lines is of the order of 2 or 3, or maybe 5
or 6.

Originally C Mode was simply a defun.  Now it's got prog-mode as a
super-mode.  You're advocating inserting another one or two (I'm a bit
confused as to which) modes into that tree.  This might be OK if those
modes had a coherent meaning (as prog-mode does), but it seems that they
do not.  _That_ is complexity.

What makes code difficult to read is always "having to look somewhere
else".  Your approach will split (the function) c-mode into two or three
fragments, making readers and maintainers have to deal with these two or
three bits.

> > In that patch, your assumptions about what AWK Mode needs in
> > c-{before,after}-change were incorrect.

> Could you help me understand what I missed?

Already dealt with.  At the very least, AWK Mode uses c-parse-state and
thus needs its before-change function.

> > There are likely other bugs in the patch too.  Remember, a previous,
> > "simple, cosmetic" change, using define-derived-mode to derive CC
> > Mode's modes from prog-mode, introduced two bugs, each of which took
> > several hours to track down.  One of these is still unfixed.

> I admit to not remembering, especially not the remaining bug due to
> using prog-mode.  Can you remind us which bug this is?

The two bugs were the double calling of the mode hooks, and the splatting
of AWK Mode syntax table by define-derived-mode defvar'ing
awk-mode-syntax-table itself.

> > It has taken me several hours to study your email and answer it.  Your
> > patch is all about rearranging the deckchairs around the cruise ship's
> > pool when the engines need looking at.

> Any package always has several problems that need fixing.  Just because
> a change doesn't fix what you see as the main problem doesn't mean it
> should be rejected.  My overall aim is to make cc-modes more "standard"
> so as to lower the entry barrier for other people.  I also want to make
> it more modular, which I thought I'd start by moving all the awk
> functionality out of the generic part of the code and into cc-awk.

As discussed above, you're swapping one problem for another.  All the
entry points to CC Mode are in cc-mode.el.  You'd lose that unity.  As
already suggested, maybe simply loading cc-awk.elc all the time would be
best.

> > It's trying to fix what's not broken.

> I'm not sure what other people on this list feel like w.r.t cc-mode, but
> personally so far my attitude has been to close my eyes and pinch my
> nose while forwarding the bugs to you because the code is so
> horrendously outlandish, messy, and outdated.

It's outlandish and messy, yes, because it's 30 years old and handles a
multiplicity of complicated languages.  Are there any other Emacs modes
which handle many variants on a theme, like C, C++, Java, ...?  Your
solution to the messiness and outdatedness is to use Emacs's newest
facilities, rendering the code base in need of heavy adaptation to run
elsewhere.  That's not a solution.

> > It would not contribute one single thing to fixing the many things in
> > CC Mode which need fixing.  On the contrary, it would act as
> > a distraction, merely consuming lots more of my time reconciling these
> > changes with the stand-alone CC Mode.

> The problem here is not my changes, ....

But the problem _is_ your changes.  You would quite happily have
committed your patch despite not realising it breaks AWK Mode.  It's not
reasonable to expect that you, the maintainer of Emacs as a whole, should
be familiar with the last nuance of every bit of code in the system.
What you want to change in CC Mode needs to be checked (just as something
I might want to change in syntax.c or font lock needs checking).

> ... it's having those two separate code bases.  There is no good reason
> to have 2 different code bases.

In times past, the CC Mode code base was the primary one, and the number
of changes to CC Mode made directly in savannah was small.  The latter
has changed in recent years.

> So let's fix this problem once and for all by merging the two and then
> make sure they can't diverge any more (i.e. by using the Emacs code as
> the place where development happens).

Then XEmacs and backward compatibility would inevitably disappear,
destroyed by the inexorable creeping use of GNU Emacs only facilities.
I'm not sure how you'd feel about the CC Mode version in savannah being a
portable one and remaining so.  You seem very unhappy about a lot of the
portable coding in CC Mode.

> > In the mean time, there are several reported bugs not being fixed, and
> > the enhancements needed for, e.g., C++11 are not progressing.  The time I
> > can spend on Emacs and CC Mode is much less than both of us would like.

> On the one hand you say that you don't have enough time to deal with
> CC-mode, and on the other you reject the rare patches sent your way.

These rare patches tend to be buggy and take a lot of time to check, and
they tend to address "non-functional" issues.  If the patch came in an
email with Subject: "Proposed fix for bug #23456", as one did within the
last fortnight (and also the one from Daniel Colascione in May), I'd be
very much happier.

> If you prefer that I keep just forwarding bug-reports to you and never
> ever again look at cc-mode, I can do that.  But if you want help, you'll
> need to be a bit more flexible.

> > What needs attending to in CC Mode is NOT its embedding within X?Emacs
> > and it's definitely NOT the lang-var system, which works just fine.  It's
> > the complexity in c-guess-basic-syntax, in c-font-lock-declarations,
> > c-forward-type, c-forward-decl-or-cast-1, and many other such functions,
> > and the difficulties this causes.

> One thing at a time.

:-)

>         Stefan

-- 
Alan Mackenzie (Nuremberg, Germany).