Re: Escaping quotes in docstrings, Was: A simple solution to "Upcoming loss of usability ..."

[Dmitry Gutov]

On 08/01/2015 04:36 AM, Paul Eggert wrote:

> We can cross that bridge if we ever need to come to it.  Such a use
> can't be the default everywhere, since not every platform supports
> colors and faces (e.g., batch diagnostics).  So we'll need something
> with quotes anyway.  And we already have code that highlights the
> contents of quotations so that may well be good enough.

It doesn't need to be the default, or the default everywhere. But for it 
to be feasible to implement at least in some places, 
substitute-command-keys shouldn't translate the quotes. It should limit 
itself to translating just the constructs it translated before, as well 
as putting `escaped' text property on some characters in the output string.

> > C code is perfectly able to use Lisp functions.
>
> I expect that some of the diagnostics are at a low level, and can't
> assume that Lisp functions are callable, and that we'll need to do some
> of this at the C level regardless.  Once it's done there, why bother
> with redoing it in Lisp?

Highlighting is performed in Lisp. Linkification is performed in Lisp.

For instance, help-make-xrefs scans the Help buffer, the contents of 
which have already passed through substitute-command-keys, for matches 
of help-xref-symbol-regexp. Without knowing which quotes were escaped, 
and which appeared in the docstring as-is, it can make wrong 
cross-references. Take this definition:

(defvar foo nil
  "Referencing `\\=`--pcase-macroexpander' macro.")

The references to the function name is not linkified, and to solve this 
problem better in general, Lisp will need to know which characters were 
escaped and which weren't. Here's a contrived example which can't be 
fixed without that:

(defun ’‘ (a b c)
  "It's called `’‘'."
  (+ 1 2 3))

Of course, we don't have any functions with curly quotes in the name 
now. But weren't some people just recently clamoring for wider use of 
Unicode in Emacs Lisp sources?

> Whether substitute-command-keys does it directly, or via calling some
> other function, is an implementation detail and it's not clear to me yet
> which is the right way to go here.

It shouldn't perform the translation at all. Not directly, nor via 
another function. Its callers can do the translation better, and maybe 
in different ways, depending on the context.