Re: Further CC-mode changes

[Stefan Monnier]

> A change to all of c-mode, c++-mode, ....., pike-mode.  There's a danger
> that the maintainer will forget awk-mode, since it's not there any more.

The same already applies to the dozen externally maintained modes that
use the cc-mode engine.  Making cc-awk more like those modes would
actually help make sure the modularization is done right (or at least,
"right enough for AWK").

> Another simplification would be simply to load cc-awk.elc along with the
> rest of CC Mode, on the grounds that the 19k file is no longer big enough
> to worry about on modern machines with gigabytes of RAM.

I don't see why that would help or what it would simplify.

>> You know why: `syntax-propertize' is used by all major modes except
>> cc-mode derived ones.  And it's more efficient, because it's more lazy.
> Lazy?  Does C-M-f, for example, trigger syntax-propertize in any way?

Not yet, no.

>> Also its correctness argument is much simpler since it doesn't rely on
>> interactions between before-change-functions, after-change-functions
>> (and font-lock), contrary to your code.
> The argument could run that because syntax-propertize doesn't take
> account of these subleties, its actions aren't always correct.  But ...
> syntax-propertize isn't documented in the Elisp manual.  It's unclear
> when it gets called (in relation to after-change-functions, for example)
> and even _if_ it always gets called (e.g. when font-lock isn't enabled).

That's because it's irrelevant: if *you* need the properties to be
applied, then call (syntax-propertize POS).  If they've already been
applied, then this will return immediately.

Since font-lock's syntactic fontification needs those properties to be
applied, it does call syntax-propertize.  Similarly,
indent-according-to-mode calls it since it's usually a good idea.
And yes, C-M-f should too, tho noone has been sufficiently bothered by
it to implement it yet.

> Adapting code to use it is thus a _massive_ amount of work - it involves
> reading the source code for syntax-propertize and friends to work out
> what it does.

No you don't: the contract is just what I stated above and you don't
need to know more than that.

> I suspect that for a mode which needs syntax-table text properties
> early in its after-change-functions, syntax-properties wouldn't work.

Suspect all you want.  But I assure you it works just fine.

>> > It's a lot of work to change,
>> It's work I did, pro bono.  Just enjoy it.
> No, you haven't.  What's the equivalent of `syntax-propertize' in XEmacs,
> or an old GNU Emacs?

Did you look at my patch?  It only uses syntax-propertize
when available.  When not, it keeps using your old code.  And the bulk of
the code is shared between the two cases.

> Right.  So I'm now going to have three alternative code flows in place of
> the one I currently have: (i) One flow for when font lock's enabled; (ii)
> one for when FL is disabled, which explicitly calls syntax-propertize
> from after-change-functions; (iii) one for Emacsen lacking
> syntax-propertize.  I've already got (iii).  I think you're asking me to
> add (i) and (ii).  This is an increase in complexity and a lot of work.

There are many different ways to handle backward compatibility.  You can
do it as above (where my position would be to simply keep the current
code for the old Emacsen that don't have syntax-propertize, so while
there are more different ways to work than you'd like the old ways have
been tested and there's no reason they should stop working, so you can
largely ignore them).
Or you can provide you own implementation of syntax-propertize.
It's actually simple.  If you prefer this route, I can do
that as well.

>> > This change wouldn't enhance functionality or maintainability.
>> Yes, it enhances maintainability of awk-mode by making it work more like
>> all other modes.  It also stops cc-mode from doing all kinds of weird
>> unrelated crap having to do with maintaining c-found-types and
>> c-state-cache, which are not used in cc-awk.
> c-state-cache IS used in AWK; it's used in all CC Mode modes and is
> fundamental to parsing braces etc.

Hmmm... I didn't notice it in my tests.  I guess it only shows up if you
really go around editing the test such that the cache becomes stale?
OK, I'll rework my syntax-propertize patch accordingly.

> OK, maybe it's not that bad.  define-derived-mode does exist in (at
> least) newer versions of XEmacs.  But it seems to mean having at least
> one "artificial" mode which will add to complexity.  define-derived-mode
> is a complicated macro, with so many implicit things, that often the only
> way to see what it does is with `macroexpand'.

The behavior I rely upon existed in define-derived-mode long before
I started I mess with it for Emacs-21.  You already have an "artificial"
parent mode.  Its hook is called c-mode-common-hook its keymap is called
c-mode-base-map, its name for c-lang-defconst is `t', its name for
c-fallback-mode' is nil.

Its standard practice in OO to have a parent which is not a "real" class
(because it doesn't have instances), but is only present so that various
classes can share a common parent.  This is just the same.

> See above.  I really can't foresee any circumstances where a CC Mode mode
> won't need before and after change functions.

syntax-propertize uses a before-change-function to "flush the cache", of
course.  After that you just call it when you need it.  E.g. If
font-lock itself doesn't call it, then we'll need to arrange to run it
before font-lock does its thing.

> How about doing it anyway?  A major mode incorporating user changes made
> in its mode hook into its current state is a reasonable thing to expect
> to do.

Yes, but usually those modes, just like CC-mode, need to do that after
setting up file-local variables.  So using hack-local-variables-hook is
really not a bad option.

>> In this particular case, the better fix would be to introduce a new
>> `mode-name-sidekick' in the mode-line-format, so we can get rid of
>> c-update-modeline altogether.
> I don't think XEmacs has the capability of running functions from mode
> line format elements.  I looked, once.

Your mode-line elements are sufficiently simple that there's no need to
run functions.  An element like:

      (c-electric-flag ("/l"
                        (c-auto-newline "a")
                        (c-hungry-delete-key "h")
                        (subword-mode "w"))
       (c-hungry-delete-key ("/h" (subword-mode "w")))
       (subword-mode "/w"))

should work just fine in XEmacs as well.

> The complexity is in having a deep inheritance tree of modes and "modes".

That lets users configure their CC-mode more easily: they can add
a single keybinding (or hook function) that'll appear in C, C++, and ObjC.

>> > There are likely other bugs in the patch too.  Remember, a previous,
>> > "simple, cosmetic" change, using define-derived-mode to derive CC
>> > Mode's modes from prog-mode, introduced two bugs, each of which took
>> > several hours to track down.  One of these is still unfixed.
>> I admit to not remembering, especially not the remaining bug due to
>> using prog-mode.  Can you remind us which bug this is?
> The two bugs were the double calling of the mode hooks, and the splatting
> of AWK Mode syntax table by define-derived-mode defvar'ing
> awk-mode-syntax-table itself.

Ah, right, then: both are fixed by my patch.  That's the advantage of
using "standard coding practices" which have been tried and tested in
many other modes already.

> As discussed above, you're swapping one problem for another.  All the
> entry points to CC Mode are in cc-mode.el.

I don't see d-mode's entry point in cc-mode.  Neither do I see
cuda-mode, dart-mode, php-mode, ...

> You'd lose that unity.

This unity is an illusion.

> As already suggested, maybe simply loading cc-awk.elc all the time
> would be best.

No, we do not want one big monolithic monster.

> multiplicity of complicated languages.  Are there any other Emacs modes
> which handle many variants on a theme, like C, C++, Java, ...?

There's SMIE, of course.  It's used for a fair number of programming
languages now (Coq, OCaml, SML, Prolog, RC, SH, Modula-2, Octave, Ruby,
CSS, Swift, GAP, Elixir).

> Your solution to the messiness and outdatedness is to use Emacs's
> newest facilities, rendering the code base in need of heavy adaptation
> to run elsewhere.  That's not a solution.

While I have spent most of the last decade focusing on code that only
runs on the latest Emacs, I have a fair bit of experience with backward
compatibility.  There are always various different ways to solve the problem.

BTW, I'd be interested to know more about the standalone package:
- what kind of emacsen is it mostly used on (e.g. is it mostly used by XEmacs
  users, or mostly Emacs users)?
- is it mostly used so as to get support for newer language features or
  is it mostly used to get new major-mode features?
- how many users are we talking about?

This said, I admit that my general position is that backward
compatibility is a burden that's not very valuable: after all people who
use still use Emacs-22 can use the CC-mode that came with Emacs-22.
So it's OK to make some effort at backward compatibility, but you
quickly get to a point of diminishing returns.

In the case of CC-mode, I think the outdated state of the code that
results from the way backward compatibility has been handled is slowing
down CC-mode's progress, so while a few users of XEmacs and Emacs-22 may
benefit a bit, it is to the detriment of most users of Emacs-24.

> But the problem _is_ your changes.  You would quite happily have
> committed your patch despite not realising it breaks AWK Mode.

FWIW, I sent it for review, in case you did not notice.

BTW, you still haven't told me what my previous (cosmetic) patches break
(although you did revert them).

>> So let's fix this problem once and for all by merging the two and then
>> make sure they can't diverge any more (i.e. by using the Emacs code as
>> the place where development happens).
> Then XEmacs and backward compatibility would inevitably disappear,
> destroyed by the inexorable creeping use of GNU Emacs only facilities.

Don't be ridiculous.  There are many packages included in Emacs where we
need to preserve backward compatibility (because we know that the code
is also distributed separately), so we know we have to be careful to
preserve backward compatibility.  Of course, we do occasionally break it
by accident, but its far from common.
Again, we have a lot of experience preserving backward compatibility
(even if reluctantly).

> I'm not sure how you'd feel about the CC Mode version in savannah being a
> portable one and remaining so.  You seem very unhappy about a lot of the
> portable coding in CC Mode.

I'm not happy about you refusing to use newer facilities.
That's not the same as being unhappy about preserving compatibility.

> These rare patches tend to be buggy and take a lot of time to check, and
> they tend to address "non-functional" issues.  If the patch came in an
> email with Subject: "Proposed fix for bug #23456", as one did within the
> last fortnight (and also the one from Daniel Colascione in May), I'd be
> very much happier.

I don't intend to fix bugs, no.  I intend to clean up the code so that
it is more accessible.  Hopefully that will also make it easier to
improve the modes and fix bugs.

What are my options: ask for your extensive test suite (which we did
already a long time ago, and haven't seen it yet), then test my changes
against it, then install my patch.  And then see it reverted because you
don't like my cosmetic changes?


        Stefan