Re: AUCTeX and "builtin latex mode" integration

[Ikumi Keita]

Hi all,

I think that most coding task is done with
feature/fix-mode-names-overlap branch. The remaining major problems I'm
aware of are the following two, the detail of which are explained below
in this message.
1. incompatibility of directory local variables
2. simple-minded reference to auto-mode-alist in emacs core
Though there are still additional minor issues[1], I hope they don't
count as practical problems.

I'd like to hear what others think about the current status, especially
whether we need major change in principle/philosophy in the branch or
not. If not, I think I can start revising the documentations in the
branch.

[Detail]
(a) The mode names change as follows:
    plain-tex-mode -> plain-TeX-mode
    latex-mode     -> LaTeX-mode
    doctex-mode    -> docTeX-mode
    context-mode   -> ConTeXt-mode
    texinfo-mode   -> Texinfo-mode
    ams-tex-mode   -> AmSTeX-mode
    japanese-plain-tex-mode -> japanese-plain-TeX-mode
    japanese-latex-mode     -> japanese-LaTeX-mode
    context-en-mode  DELETED
    context-nl-mode  DELETED
    TeX-tex-mode     UNCHANGED
    TeX-mode         NEW
(a-1) New mode names are chosen to match the existing variables, so most
      user codes would continue to work without modification. For
      example, names of keymaps and hooks don't change.
      (`AmS-TeX-mode-hook' is renamed to `AmSTeX-mode-hook', but
      compatibility alias is provided by `define-obsolete-variable-alias'.)
(a-2) Now `TeX-add-local-master' adds entry of new mode names such as
%%% Local Variables:
%%% mode: LaTeX      <-- not `latex'
%%% End:
(a-3) The mode aliases with `TeX-' prefix, e.g. `TeX-latex-mode', are
      deleted except `TeX-tex-mode'.
(a-4) TeX-mode is meant for the base mode for other major modes. Its
      role is to provide base keymap, hook and syntax table under the
      same name with the current AUCTeX and run the initialization code
      previously provided as `VirTeX-common-initialization'.
      VirTeX-common-initialization no longer exists. It was absorbed
      into major mode function `TeX-mode'.

(b) All new modes (except TeX-tex-mode) are defined by
    `define-derived-mode'. The inheritance relations are:

text-mode      --+-- TeX-mode
                 +-- Texinfo-mode

TeX-mode       --+-- plain-TeX-mode
                 +-- LaTeX-mode
                 +-- ConTeXt-mode

plain-TeX-mode --+-- AmSTeX-mode
                 +-- japanese-plain-TeX-mode

LaTeX-mode     --+-- docTeX-mode
                 +-- japanese-LaTeX-mode

     This change introduces the following differences:
(b-1) Now that all modes (except TeX-tex-mode) has `text-mode' as their
      ancestor, they inherit keymap and syntax table of `text-mode'.
      (I added a piece of code to suppress "Text" entry in the menu bar,
       defined in `text-mode-map'.)

      (However, Texinfo-mode is exceptional in the following aspects:
        (b-1-1) It doesn't inherit text-mode-syntax-table because it
                simply uses built-in mode's texinfo-mode-syntax-table, 
                which is independent of text-mode-syntax-table. This
                situation is the same with the current Texinfo mode.
        (b-1-2) Texinfo-mode-map has TeX-mode-map as its direct parent.
                This is the same with the current Texinfo mode. Now it
                inherits text-mode-map indirectly through TeX-mode-map.

       In addition, AmSTeX-mode-map still has TeX-mode-map as its parent
       and doesn't inherit plain-TeX-mode-map; this is necessary not to
       mess up the menu bar in AmSTeX mode. See the relevant codes in
       plain-tex.el for detail.)
(b-2) AmSTeX-mode runs plain-TeX-mode-hook.
(b-3) There are new keymaps, hooks and abbrev tables:
      Texinfo-mode-abbrev-table
      japanese-plain-TeX-mode-map, japanese-LaTeX-mode-map
      japanese-plain-TeX-mode-hook, japanese-LaTeX-mode-hook
      japanese-plain-TeX-mode-abbrev-table,
      japanese-LaTeX-mode-hook-abbrev-table

      At last we have abbrev table in Texinfo mode.

(c) The compatibility with the former mode names with respect to
    function call are retained by redirections or aliases.
(c-1) Former names which overlap with built-in modes, namely
      plain-tex-mode, latex-mode, doctex-mode, texinfo-mode and tex-mode
      are handled by redirections; the same override advices as before
      are continued to used for emacs<29 while `major-mode-remap-alist'
      is used for emacs 29 (and later).
      (Therefore, if there are user code which call `latex-mode'
       directly, built-in latex-mode runs instead of AUCTeX LaTeX-mode
       in emacs 29.)
(c-2) Other former names, e.g. context-mode and japanese-latex-mode, are
      handled by aliases such as `(defalias 'context-mode
      #'ConTeXt-mode)'.

[Incompatibility]
1. directory local variables
  Currently, directory local variables are prepared in a entry like
 (latex-mode . ((reftex-label-alist . (("equation" ?e "eq:%f-" "\\eqref{%s}")
                                       ("align" ?e "eq:%f-" "\\eqref{%s}")
                                       ("gather" ?e "eq:%f-" "\\eqref{%s}")))))
  in .dir-locals.el. In this feature branch, mode name is `LaTeX-mode',
  thus this entry is ignored.

  The facility of major-mode-remap-list still does not handle this
  situation.

  We can announce this fact and ask users to rewrite their
  .dir-locals.el, but that can't be done if the user doesn't have write
  access for that particular .dir-locals.el. (Maybe we can ignore such
  corner cases?)

2. simple-minded reference to auto-mode-alist in emacs core
  As discussed in [2], when elisp codes look at auto-mode-alist
  directly and compare its content with `major-mode', new mode names
  such as `LaTeX-mode' get no hits and lead to failure.
  I added workaround for cedet/semantic/symref/grep.el, but similar
  problematic code still exists at least in cedet/semantic/idle.el:
----------------------------------------------------------------------
(defun semantic-idle-scheduler-work-parse-neighboring-files ()
...(snip)...
    ;; Collect all patterns matching files of the same mode we edit.
    (mapc (lambda (pat) (and (eq (cdr pat) major-mode)
                             (push (car pat) matching-auto-mode-patterns)))
          auto-mode-alist)
----------------------------------------------------------------------

Regards,
Ikumi Keita
#StandWithUkraine #StopWarInUkraine

[1] items a-1, c discussed in 
https://lists.gnu.org/r/auctex-devel/2023-05/msg00004.html
[2] https://lists.gnu.org/r/auctex-devel/2023-09/msg00002.html
    https://lists.gnu.org/r/auctex-devel/2023-09/msg00004.html