Re: Modern conventions for structuring Emacs Lisp libraries

[Stefan Monnier]

> as a second take on this topic, I would like to make the attached
> proposal for improving the conventions for structuring Emacs Lisp
> source-code files:

Thanks.  I think it'd be good to improve the current situation, but
I don't think anything more than refinement is an option.  IOW changing
the convention is not really an option for various reasons, such as the
fact that several tools rely on it, and there are so many files using
it, many of which we can't fix, that we'd be stuck with two different
conventions for the next many years.

> 2.2.1 Last Line Pathology
> -------------------------

This is a very minor issue, I think.  It's trivil to fix it by simply
ignoring any "trailing header".

> 2.2.2 Wasted First Level Weakness
> ---------------------------------
>   | 7 matches for "^;;; " in buffer: help.el.gz
>   |       1:;;; help.el --- help commands for Emacs
>   |      25:;;; Commentary:
>   |      30:;;; Code:
>   |     280:;;; `User' help functions
>   |     967:;;; Automatic resizing of temporary buffers.
>   |    1041:;;; Help windows.
>   |    1193:;;; help.el ends here
>   `----
>   The above example, produced by typing '1' in the associated
>   *Navi-buffer* of help.el (-> show headlines up-to first-level), shows
>   this weakness quite clearly:
>   1. in a file with some 1200 lines of code, there are 7 1st-level
>      headlines
>   2. at most 3 (if not only 2) of 7 headlines should really be 1st-level
>      headlines:

Again, this seems like a rather minor issue.  Furthermore, ";;; Code:"
is a well-known header, so if needed, we can special case it in code
such as navi.

>   all other headlines in the header-comment section are subheadlines of
>   this one, i.e. level 2 or higher.

You can get that without any modification to existing files by
considering that any header that comes before ";;; Code:" is really one
deeper than what its semi-colons say.

>   In oldschool Elisp files, headlines often end in colons. In Org-mode,
>   this is not usual, and the Org-mode conventions are better in this
>   case.

Again, trivial to "fix" in any tool like navi: just change a trailing
colon into a full stop.

>   The otherwise fantastic library dired.el illustrates why:

All those non-headers are bugs.

>   8) shows a wild mix of real headlines and comments. This is because it
>   is only natural for people to get creative with comment characters

Maybe it was a bad idea to choose this convention, but it's easier to
fix the few cases where people's use of comments ends up accidentally
colliding with your convention.

>   |       1:;; * iorg-scrape.el --- elisp glue code for 
> `picoLisp/lib/scrape.l'

Note that ";; *" is already in common use for other purposes (basically
the * is used as a bullet point there), so your convention would also
bump into other problems.

>   A big plus: this is /major-mode agnostic/. All 3 libraries (outshine,

I'm pretty sure there are many other languages that define their own
notion of comment structure, and I'd be surprise if any of them happens
to choose your convention.  IOW your convention is "major-mode agnostic"
at the cost of not following the "normal" conventions.  Just as is the
case with allout.el.

This is a trade-off with its benefit and its downsides.

The other option is to make your tools's structuring convention
sufficiently customizable that it can make adapt to the
"normal" convention (this is the path taken by outline.el).  It's not
perfect either, of course, but I think that given the obstacles it's
either to adapt the tools than to change the conventions.

>   Conversion between oldschool and outshine headlines would be
>   exceedingly easy - as long as the oldschool files don't mix comments
>   and headlines, i.e don't have "invalid" syntax (in a loose sense).

Exactly: if they're correct, then the conversion is easy.  So easy that
you can do it on the fly, without touching the file! ;-)

>   | ;; * iorg-scrape.el --- elisp glue code for `picoLisp/lib/scrape.l'
>   | ;; ** MetaData
>   | ;;   :PROPERTIES:
>   | ;;   :copyright: Thorsten_Jolitz
>   | ;;   :copyright-since: 2013

IIUC the syntax of the "copyright" line is legally significant
(e.g. whether years can be abbreviated, whether year ranges can be used,
etc...), and I don't think I want to get into discussions about what
other syntax we can use.  So I'm not interested in changing the
copyright line.

>   | ;;   :version:  0.9

The ";; Version:" format is used in ELPA, so changing it would require
changes in too many places.  Not gonna happen.

>   | ;;   :licence:  GPL3+
>   | ;;   :licence-url: http://www.gnu.org/licenses/

Here again, the many-lines blurb about the license has legal
significance (otherwise we'd have switched to ";; License: GPLv3+" by
now).

>   2. Since a [Property API] for Org-mode's properties exists, reading
>      and writing them from an Emacs Lisp program becomes almost trivial.

Reading them is already trivial (via the lisp-mnt.el library).
Writing is usually very easy already; the only exception could the
"Keywords:" header which is just a big problem in itself anyway (we
don't really know which categories should be available).

> 3 Summary
> =========

Damn!  You skipped the part I was looking for: the "Commentary:"
section.  I do want this part to be refined.  More specifically, I'd
like someone to come up with a description of a markup format to use
there (99% compatible with what we already have), together with code
that can turn such a Commentary section into nicely rendered text in an
Emacs buffer.


        Stefan