bug#27270: display-raw-bytes-as-hex generates ambiguous output for Emacs strings

[Paul Eggert]

On 06/08/2017 09:11 AM, Eli Zaretskii wrote:
> > (setq display-raw-bytes-as-hex t) C-j
> > (format "%c%c" ?\u0090 ?5) C-j
> >
> > Emacs displays this:
> >
> > "\x905"
> >
> > which is the wrong string visually.
> How is that different from "\2205" you get under the default settings?

When I cut and paste "\2205" into another Emacs, it evaluates to the 
same two-character string that I started off with because octal escapes 
are limited to 3 octal digits. When I cut and paste "\x905" I get a 
one-character string because there is no limit to the length of 
hexadecimal escapes. This is a problem, because cut-and-paste should 
continue to copy text accurately even when I'm using terminal windows.

> > The string should be
> > displayed unambiguously, either like this:
> >
> > "\x80\ 5"
> >
> > or via some other means.
> We do use "some other means": the raw byte has a different face.

That doesn't help when --color=no is specified, or in terminal sessions 
that do not support colors. And the colors, even when present, do not 
survive cutting and pasting, which copies the text without colors. So 
this is a real problem.

> But if you evaluate the above in*scratch*, you won't see that because of
> font-lock.  Turn off font-lock-mode, and you will clearly see where
> the raw byte ends and "normal" text begins.

Turning off font-lock-mode doesn't help when colors are disabled. I 
often run with colors disabled, since my terminal color scheme disagrees 
with Emacs's and I prefer monochrome anyway. So this ambiguity will be a 
real pain for me.