Re: Emacs Lisp's future

[Eli Zaretskii]

> Date: Wed, 08 Oct 2014 21:19:54 -0400
> From: Richard Stallman <address@hidden>
> CC: address@hidden, address@hidden, address@hidden,
>       address@hidden, address@hidden, address@hidden,
>       address@hidden
> 
>     We do ask, but only at buffer save time.  Asking questions when
>     visiting a file is perceived as a nuisance, because our heuristics
>     that detect these cases are imperfect and tend to have high enough
>     false positive rate that annoys people.
> 
> Asking about invalid UTF-8 in a file could be a nuisance, but how much
> of a nuisance depends on the details of what we do.  Since this has
> some security implications, it is worth a small amount of nuisance.

That wasn't what users felt, overwhelmingly.

> What exactly did we try before?

AFAIR, we tried converting raw bytes into valid non-ASCII characters,
and perhaps also replacing them with the equivalent of u+FFFD, the
Unicode "replacement character".

> Meanwhile, in a separate message I wrote about non-file operations:

Well, you said "frile", which confused me ;-)  However, ...

>     > We can set the defaults for those non-frile interfaces so as to reject
>     > invalid UTF-8 sequences.  Then a program could specify to override the
>     > default and allow them.
> 
>     That has been tried (not with UTF-8, but I don't think this matters),
>     and failed miserably.
> 
> I don't think we are talking about the same thing.  I am talking about Lisp
> functions to do conversions on text that does NOT come from files.

... Emacs treats all of these cases the same.  For text we are going
to send to a process or network stream, we ask the above-mentioned
question at the time we encode the internal representation into the
external byte stream we are about to send.  E.g., you can see that in
action in sending mail if you insert some raw bytes into a mail
message in a *mail* buffer, and then try sending it.  There's no file
involved here, at least not as far as Emacs is concerned, and yet you
will see the same prompt asking you to select a proper encoding.

> You seem to be talking about operations on files:
> 
>                          The experience taught us that Emacs users
>     definitely don't want Emacs to do _anything_ about the unmodified
>     parts of text, except copy it verbatim.  Even the question we ask at
>     buffer-save time is sometimes reported as an annoyance.
> 
> It looks like you're grouping the two cases together,
> while I am treating them separately.

Emacs treats them both the same way, and uses the same low-level
primitives that generally don't know the purpose of the byte stream
they are encoding or decoding.  All they know is whether the source
resp. destination is a buffer, a string, or a gap in the buffer text,
which is insufficient to distinguish between the 2 use cases you are
trying to treat separately.