bug#27544: 25.1; Visualization of Unicode bidirectional marks

[Itai Berli]

A default should be something that aligns with the expectations of a
casual user; you cannot expect a casual user to start writing elisp
code or tinkering with `glyphless-char-display-control`.

And how can we know what a casual user would expect?

1. We can follow established standards, particularly if we claim to
conform to them, as Emacs does to the Unicode standard. The Unicode
bidi algorithm 8.0.0 specifications state the following about LRM, RLM
and ALM (section 2.6 Implicit Directional Marks):
> they do not display or have any other semantic effect
and again, in the same section:
>  they do not appear in the display

2. We can see how other text editors behave. The following is a sample
of text editors that are bidi-aware, and none of them displays
explicit bidi marks (they only support RLM and LRM), not even as a
very thing space: Gmail, Google Docs, Libre Writer, MS-Word, Pages,
and TextEdit. By the way, the Unicode bidi algorithm was written by
three editors, two of whom work for Google, and the other one - for
Microsoft, so seeing how Google docs and MS-Word implement
bidirectionally is a good way of seeing how the standard should be
implemented.

3. We can employ common sense. Either the Emacs user will notice the
spaces introduced by the bidi marks, or he won't.

If he does, then it will be visually annoying for him, and will thwart
attempts to realign and position the text precisely.

If he doesn't notice the spaces, then he won't understand, for
instance, why he needs to press the left arrow four times to position
the cursor past the first visible character of a paragraph that begins
with three bidi marks.

In both cases, he won't be able to see the difference between the
various marks, which is a cause for confusion, frustration, and
errors.

4. We can see how Emacs implements similar constructs. The bidi marks
are control sequences that can be inserted into the editor using `C-x
8 RET <Unicode hex number>`. Let's see how other control sequences are
handled by this mechanism. If you try to insert, say, the characters
Null character (U+0000), Bell character (U+0007) and Escape character
(U+001B) using this method, we'll get:

ILLUSTRATION: The characters Null character (U+0000), Bell character
(U+0007) and Escape character (U+001B), in this order.
http://imgur.com/bsJoblk

We see that the output satisfies the three condition I specified in my
original post: (a) Easy to notice, (b) Distinguishable from other
whitespace visualization glyphs, (c) Distinct from one another.

However, in light of to this example I should state that, in my
opinion, unlike other control characters, Emacs should not display the
bidi marks unless in `whitespace` mode. The difference is that control
characters are rarely used by casual writers of alphabetic text,
whereas explicit bidirectional marks are quite commonly used.