[Orgmode] extensible syntax

[Samuel Wales]

A general idea, which might or might not be useful.

There are occasionally questions about syntax, like this:

  Also, I'm afraid definition matching regexp won't play
  nicely with text indentation, ... -- Paul

Or this:

  What would be safer?  -- Carsten

I like the footnote implementation, so this is for future
features, not necessarily footnotes.

One issue when implementing new syntax (or changing existing
syntax or cleaning up code) is parsing risk, which I will
define as the risk that the syntax and the regexp or
matching code:

  1) conflicts with user text
  2) conflicts with existing features
  3) will be hard to maintain
  4) constrains future features by making them conflict
     syntactically
  5) makes you run out of syntax to use in the future
  6) will require complicated regexps
  7) doesn't readily handle stuff you might want in the
     future, like being combined with another feature
  8) will be hard to quote, escape, comment, *boldify*, etc.
  9) doesn't handle nestability, print-readability,
     pretty-printability, syntax coloring, etc.
  10) will be inefficient when called in a loop
  11) isn't factored out
  12) etc.

For example, one of the many reasons for using org-IDs (:))
in the conversation manager (as proposed) is that there are
already functions to parse org-IDs, so a new syntax is not
necessary and therefore parsing risk is minimized.

In case parsing risk is a concern when adding a new feature
to org, here is one idea: have a generic syntax that is
extensible with keywords.

The idea is to have a bracketing syntax with a reserved
keyword as the first element that says what you are doing.

To use footnotes as an example (this is not a suggestion to
change footnote syntax, just an example that can be used for
future syntax):

You might use something like "here is some text to be
footnoted $[fn apple]" to specify the footnote link, and
"$[fn-target apples are delicious]" to specify the target.

The general point I want to make is that once such a syntax
is decided on, many future features are possible without
introducing parsing risk.

For example, you can implement a new feature as
"$[my-new-feature ...]".  Then there is no parsing risk,
even though you have just added a new feature.

For modifications of features, you can use keywords:
"$[my-new-feature ... :myparameter ...]".  These are easily
parsed by standard functions for parsing lists.

You can develop ways to boldify, quote, nest, prettily
print, etc. this syntax, and those ways will work well with
all future features that use it.

Of course, the dollar sign and brackets are not the only
possibility; it could be percent sign and parentheses, for
example.

You will not be starting from scratch.  Lisp has already
worked out many of these issues.

I will leave it to those who write massive amounts of org
code to decide whether an extensible syntax might be useful
to reduce parsing risk for future features.

But I thought that I would propose the idea in case it is of
interest.

-- 
For personal gain, myalgic encephalomyelitis denialists are knowingly
causing further suffering and death by grossly corrupting science.  Do
you care about the world?
http://www.meactionuk.org.uk/What_Is_ME_What_Is_CFS.htm