Re: Bidirectional text and URLs

[Eli Zaretskii]

> Date: Mon, 01 Dec 2014 05:17:58 -0500
> From: Richard Stallman <address@hidden>
> CC: address@hidden, address@hidden
> 
> We need to make Emacs safe and clear for users who don't know anything
> about bidi and don't want to.

I think we are in violent agreement here.  The question is how to do
that, not whether or not do it.

> One idea: change the mode line color when there is any RTL text
> (in the buffer, or on the screen, whichever is easier).

That's possible, but I think it's too drastic.  Just having RTL text
doesn't yet constitute any danger or require special vigilance on the
part of the user, even if she doesn't want to know anything about
bidi, let alone if she does.  And of course, the display engine only
examines the visible portion of the buffer and sometimes a small
region above and below, so it cannot really tell what's in the rest of
the buffer.

OTOH, we have indications on the mode line, such as "(DOS)", which
users in the past said they didn't pay attention to.  My conclusion
from that is that mode-line indication is only effective when we know
users will look at the mode line at the right moment.

> Another idea: make magic bidi characters visible by default.  People
> who edit in RTL languages and get used to bidi could set a user option
> to make them invisible.

This is both possible and easy, we already have infrastructure for
this.  Not sure it's enough, though: the reordering effect on URLs,
like in the example that started this thread, will still be there, and
seeing the actual URL where the link will take the user if clicked
upon will still be not easy enough, IMO.

>     > This is the first time I've observe RTL display in Emacs.  I don't see
>     > any way to detect the magic character that specifies it.
>
>     That's because there isn't one, in the citation you provided.
> 
> Yes there was -- you said so yourself:
> 
>   > where there is a u+202e character

There was no such character in your mail, only in the one sent by
Lars.  So I assumed you somehow lost it.  My bad.

>   > > I think we need to provide a way to make them visible.
> 
>   > We already have it: the glyphless-char-display char-table.
> 
> We need a convenient _user-level_ feature to make them visible.

We have glyphless-char-display-control, which is a defcustom.  If that
is still too technical, we can have a minor mode to set that for these
directional controls, or maybe just for some subset of them (most of
them cannot cause such disastrous effects on display).

>   > I don't think so: these controls should normally be all but invisible.
> 
> We need to make it easy to see them.  Otherwise people can't tell why
> strangeness is happening on their screens.

I think we should prefer making them visible only in the context where
they could cause harm.  Making them visible everywhere could be an
annoyance.

>   > > Also, is there a way to disable bidi in the current buffer?
>   > > If not, I think we need one.
> 
>   > There is a way, but it is not meant for Lisp programs, only for
>   > debugging the display engine.
> 
> It needs to be made convenient for users.

I don't think this is needed.  People who don't read and don't
understand about bidi will not find it useful, because they cannot
read text affected by the reordering anyway, regardless of its order.

This could only help in the rare situations such as the one discussed
here.  But in those cases, I think we all agree that Emacs should
detect them and act on them automatically; passing the buck to the
user would be a mistake on our part.

But even if I'd agree with you, making a convenient and reliable way
of going back to unidirectional display of Emacs 23 and before would
require a lot of work, because the current display engine no longer
supports unidirectional display without reordering, at least not
reliably.  The old unidirectional code was left in some of the places,
either as a debugging aid or for special corner cases, like unibyte
buffers.  In other places, the code was simply rewritten to work only
through the reordering engine, and the old code no longer exists.  For
example, display strings and overlay strings are rendered exclusively
by the reordering engine.

IOW, the unidirectional display code is for all practical purposes
gone; what's left is not reliable enough for users to use it.  So we
simply cannot turn reordering off and get an otherwise the same Emacs.

> I don't speak any RTL language (and those characters won't display on
> this tty anyway).  So I never see bidi at work, or at least not in a
> way I would notice.  I get mail that might be in Arabic script, but
> that's just a guess.  The messages are spam, so I delete them.

Once again, the only dangerous situation with bidi we are aware of is
the one that started this thread: a malicious use of directional
overrides that changes the visual appearance of what is otherwise
strict left-to-right text.  Let's concentrate on solving this rather
unique situation.  It is IMO wrong to try to generalize these rare
cases into a view that bidi reordering is somehow a menace that users
need to turn off every now and then; it isn't.

> We need to make Emacs safe and clear for users who don't know anything
> about bidi and don't want to.

Again, I think we are in violent agreement here.  The question is how to do
that, not whether or not do it.

But disabling bidi is not the way.

Several useful ideas were raised in this discussion.  I suggest that
we implement some of them and see if they are enough.