bug#21780: 25.0.50; Saving *Help* results in bad encoding because of curly quotes

[Eli Zaretskii]

> Date: Thu, 29 Oct 2015 10:58:24 -0700 (PDT)
> From: Drew Adams <drew.adams@oracle.com>
> Cc: 21780-done@debbugs.gnu.org
> 
> > > emacs -Q
> > > M-x load-library isearch.el
> > > C-h f isearch-forward
> > > In buffer *Help*: C-x C-w foo.txt
> > >
> > > You get a coding-system warning.  I tried saving it as utf-8 and as raw
> > > text.
> > >
> > > In both cases, when I open that file in a new Emacs session, I see octal
> > > escapes where there were curly quotes.
> > 
> > Thanks, I fixed the first part of this: Emacs should no longer ask
> > annoying questions when you save help buffers with curved quotes.
> > 
> > The second part, which happens when visiting the saved file, is not a
> > bug: you need to specify the encoding of files when visiting them in
> > locales whose default encoding is different.  (Actually, I expect this
> > to work automatically for you, at least in "emacs -Q", but that
> > doesn't happen in every locale.)
> 
> I guess I should interpret this as meaning that the bug is fixed (?).

Yes, I think so.

> But I don't understand the second part.  What do I need to change, as
> a user, to get this to work as I would expect?

It might work as you expect already.  You can try this:

  . After "C-h f some-function RET", switch to the *Help* buffer and
    type "C-x RET f utf-8 RET", then save the buffer as in your
    recipe.

  . Now visit the file where you saved the *Help* buffer: if the
    curved quotes display correctly, then "it works for you".

  . If the curved quotes look like raw bytes or, worse, pairs of
    non-ASCII characters, you need to visit such file like this:

      C-x RET c utf-8 RET C-x C-f FILE-NAME RET

> In Emacs, before saving, the buffer looks fine.

It looks fine, but the encoding mnemonic on the mode line is not "U"
(which stands for UTF-8), right?  That is why Emacs asks you for
encoding: it cannot save these characters using your locale's default
encoding (which is what the *Help* buffer uses by default).

> When visiting the resulting file it does not look right - it is
> unreadable.  There are 3 octal escapes for each opening curly quote
> and 3 of them for each closing curly quote.  That can amount to
> quite a lot of noise.

Yes, because you probably told Emacs to use raw-text or somesuch, when
it asked.

> Do I need to save the buffer using some other encoding?  If so, which?

Yes, you could tell it to use UTF-8 when it asked.  (After my change,
Emacs will do that automatically, no questions asked, when saving
*Help* buffers with curved quotes.)

> Emacs proposed two encodings (one of which was raw text, which I tried;
> and I tried also utf-8, which I would have thought would show curly
> quotes OK.

UTF-8 should have worked.  I wouldn't expect you to see octal escapes
after saving in UTF-8.

> I would think that Emacs would DTRT when opening the file, based on
> the encoding used to save it.

It cannot always do that.  To make sure it always does, there should
be a 'coding' cookie in the file or a file-local variable to the same
effect.  But you will have to add it manually; I don't think it's OK
for Emacs to insert such additions on its own, because Emacs has no
idea how the file will be used.

> Should users really need to do something special each time they
> visit the file?  They've never had to do this before, for basic,
> common *Help* output.

If you customize text-quoting-style to use ASCII characters for
quoting, Emacs will still behave as it did before: the file you
produce will be pure ASCII, so no decoding is necessary.

> If readers have to jump through hoops (e.g. changing "locales"),
> and there is no good fix for this regression in behavior, then I'd
> suggest that maybe `describe-*' commands should not use curly quotes.

Saving a *Help* buffer is not a frequent operation, and most users
nowadays live in UTF-8 locales anyway.  And even in some non-UTF-8
locales Emacs will succeed in displaying the file correctly when
visiting it, even without the need to type "C-x RET c".  So I think
this is not a catastrophe.

> [Or could this perhaps be a font problem?  Might the default font
> (e.g. on MS Windows) just need to be changed?]

No, it's not a font problem: Emacs did display those characters before
you saved the buffer, right?