From: Patrick Xarles Hayes <
pxhayes@gmail.com>
To:
bug-gnu-emacs@gnu.orgSubject: 24.1; 24.1; Emacs doesn't correctly read the code points U+D800–U+DFFF when opening a file
--text follows this line--
This bug report will be sent to the Bug-GNU-Emacs mailing list
and the GNU bug tracker at
debbugs.gnu.org. Please check that
the From: line contains a valid email address. After a delay of up
to one day, you should receive an acknowledgement at that address.
Please write in English if possible, as the Emacs maintainers
usually do not have translators for other languages.
Please describe exactly what actions triggered the bug, and
the precise symptoms of the bug. If you can, give a recipe
starting from `emacs -Q':
$ emacs -Q
C-x C-f
Find file: some-file-with-code-points-in-the-range-U+D800–U+DFFF
Emacs 24.1.1 loads the file interpreting any code point in the range of
U+D800–U+DFFF as a sequence of three bytes (e.g., “\355\240\200” instead
of a glyph for U+D800). These are the code points in: Non Private Use
High Surrogate, First; Non Private Use High Surrogate, Last; Private Use
High Surrogate, First; Private Use High Surrogate, Last; Low Surrogate,
First; and Low Surrogate, Last.
If one tries to save the file above, Emacs throws a warning such as the following:
<blockquote>
These default coding systems were tried to encode text
in the buffer `π-UnicodeData-6.2.0.txt':
(utf-8-unix (935870 . 4194285) (935871 . 4194208) (935872 . 4194176)
(935945 . 4194285) (935946 . 4194221) (935947 . 4194239) (936019
. 4194285) (936020 . 4194222) (936021 . 4194176) (936090 . 4194285)
(936091 . 4194223))
However, each of them encountered characters it couldn't encode:
utf-8-unix cannot encode these: ...
Click on a character (or switch to this window by `C-x o'
and select the characters by RET) to jump to the place it appears,
where `C-u C-x =' will give information about it.
Select one of the safe coding systems listed below,
or cancel the writing with C-g and edit the buffer
to remove or modify the problematic characters,
or specify any other coding system (and risk losing
the problematic characters).
raw-text emacs-mule no-conversion
</blockquote>
Emacs does, however, save the file―I think even correctly―after one
specifies a coding system for the save such as utf-8 or raw-text.
One can successfully insert characters in the range U+D800–U+DFFF and
also successfully save the file without warning. However, when one
closes and then reopens that file, the same problem as above happens.
If Emacs crashed, and you have the Emacs process in the gdb debugger,
please include the output from the following gdb commands:
`bt full' and `xbacktrace'.
For information about debugging Emacs, please read the file
/usr/share/emacs/24.1/etc/DEBUG.
In GNU Emacs 24.1.1 (x86_64-pc-linux-gnu, GTK+ Version 2.24.12)
of 2012-09-22 on batsu, modified by Debian
Windowing system distributor `The X.Org Foundation', version 11.0.11300000
Configured using:
`configure '--build' 'x86_64-linux-gnu' '--build' 'x86_64-linux-gnu'
'--prefix=/usr' '--sharedstatedir=/var/lib' '--libexecdir=/usr/lib'
'--localstatedir=/var/lib' '--infodir=/usr/share/info'
'--mandir=/usr/share/man' '--with-pop=yes'
'--enable-locallisppath=/etc/emacs24:/etc/emacs:/usr/local/share/emacs/24.1/site-lisp:/usr/local/share/emacs/site-lisp:/usr/share/emacs/24.1/site-lisp:/usr/share/emacs/site-lisp'
'--with-crt-dir=/usr/lib/x86_64-linux-gnu' '--with-x=yes'
'--with-x-toolkit=gtk' '--with-toolkit-scroll-bars'
'build_alias=x86_64-linux-gnu' 'CFLAGS=-g -O2 -fstack-protector
--param=ssp-buffer-size=4 -Wformat -Werror=format-security -Wall -O2'
'CPPFLAGS=-D_FORTIFY_SOURCE=2''
Important settings:
value of $LC_ALL: nil
value of $LC_COLLATE: POSIX
value of $LC_CTYPE: nil
value of $LC_MESSAGES: nil
value of $LC_MONETARY: nil
value of $LC_NUMERIC: nil
value of $LC_TIME: nil
value of $LANG: en_US.UTF-8
value of $XMODIFIERS: @im=ibus
locale-coding-system: utf-8-unix
default enable-multibyte-characters: t
Major mode: Help
Minor modes in effect:
tooltip-mode: t
mouse-wheel-mode: t
tool-bar-mode: t
menu-bar-mode: t
file-name-shadow-mode: t
global-font-lock-mode: t
font-lock-mode: t
blink-cursor-mode: t
auto-composition-mode: t
auto-encryption-mode: t
auto-compression-mode: t
line-number-mode: t
transient-mark-mode: t
Recent input:
C-x C-f <return> <down> <down> <down> <down> <down>
<down> <return> C-s d 9 8 0 <backspace> <backspace>
<backspace> 8 0 0 C-a C-x C-w <C-Multi_key> C-a C-k
/ t m p / t m p 2 . t t <backspace> x t <return> u
t f C-g ; l <backspace> <backspace> C-x p C-x o C-x
1 ; l ; C-x b <return> C-x b <return> h <help-echo>
; l ; l <next> <prior> C-x o <next> <next> <next> <next>
<next> <next> <home> C-x 0 ; l <up> <up> <up> <up>
<up> <up> <up> <up> <up> <up> <up> <up> <up> C-SPC
M-> M-w M-x r e p o r t - e m a <tab> <return>
Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.
Mark saved where search started
Saving file /tmp/tmp2.txt...
Quit
Auto-saving...done
Type C-x 1 to delete the help window, C-M-v to scroll help.
Mark set
Load-path shadows:
/usr/share/emacs/24.1/site-lisp/debian-startup hides /usr/share/emacs/site-lisp/debian-startup
/usr/share/emacs/24.1/site-lisp/cdargs hides /usr/share/emacs/site-lisp/cdargs
/usr/share/emacs/24.1/site-lisp/flim/md4 hides /usr/share/emacs/24.1/lisp/md4
/usr/share/emacs/24.1/site-lisp/flim/hex-util hides /usr/share/emacs/24.1/lisp/hex-util
/usr/share/emacs/24.1/site-lisp/dictionaries-common/ispell hides /usr/share/emacs/24.1/lisp/textmodes/ispell
/usr/share/emacs/24.1/site-lisp/dictionaries-common/flyspell hides /usr/share/emacs/24.1/lisp/textmodes/flyspell
/usr/share/emacs/24.1/site-lisp/prolog-el/prolog hides /usr/share/emacs/24.1/lisp/progmodes/prolog
/usr/share/emacs/24.1/site-lisp/org-mode/org-protocol hides /usr/share/emacs/24.1/lisp/org/org-protocol
/usr/share/emacs/24.1/site-lisp/org-mode/org-ascii hides /usr/share/emacs/24.1/lisp/org/org-ascii
/usr/share/emacs/24.1/site-lisp/org-mode/ob-mscgen hides /usr/share/emacs/24.1/lisp/org/ob-mscgen
/usr/share/emacs/24.1/site-lisp/org-mode/ob-sqlite hides /usr/share/emacs/24.1/lisp/org/ob-sqlite
/usr/share/emacs/24.1/site-lisp/org-mode/org-w3m hides /usr/share/emacs/24.1/lisp/org/org-w3m
/usr/share/emacs/24.1/site-lisp/org-mode/org-freemind hides /usr/share/emacs/24.1/lisp/org/org-freemind
/usr/share/emacs/24.1/site-lisp/org-mode/org-pcomplete hides /usr/share/emacs/24.1/lisp/org/org-pcomplete
/usr/share/emacs/24.1/site-lisp/org-mode/ob-shen hides /usr/share/emacs/24.1/lisp/org/ob-shen
/usr/share/emacs/24.1/site-lisp/org-mode/org-faces hides /usr/share/emacs/24.1/lisp/org/org-faces
/usr/share/emacs/24.1/site-lisp/org-mode/ob-calc hides /usr/share/emacs/24.1/lisp/org/ob-calc
/usr/share/emacs/24.1/site-lisp/org-mode/org-plot hides /usr/share/emacs/24.1/lisp/org/org-plot
/usr/share/emacs/24.1/site-lisp/org-mode/org-mac-message hides /usr/share/emacs/24.1/lisp/org/org-mac-message
/usr/share/emacs/24.1/site-lisp/org-mode/org-habit hides /usr/share/emacs/24.1/lisp/org/org-habit
/usr/share/emacs/24.1/site-lisp/org-mode/org-attach hides /usr/share/emacs/24.1/lisp/org/org-attach
/usr/share/emacs/24.1/site-lisp/org-mode/org-docview hides /usr/share/emacs/24.1/lisp/org/org-docview
/usr/share/emacs/24.1/site-lisp/org-mode/ob-eval hides /usr/share/emacs/24.1/lisp/org/ob-eval
/usr/share/emacs/24.1/site-lisp/org-mode/ob-ruby hides /usr/share/emacs/24.1/lisp/org/ob-ruby
/usr/share/emacs/24.1/site-lisp/org-mode/ob-fortran hides /usr/share/emacs/24.1/lisp/org/ob-fortran
/usr/share/emacs/24.1/site-lisp/org-mode/ob-sh hides /usr/share/emacs/24.1/lisp/org/ob-sh
/usr/share/emacs/24.1/site-lisp/org-mode/org-bbdb hides /usr/share/emacs/24.1/lisp/org/org-bbdb
/usr/share/emacs/24.1/site-lisp/org-mode/ob-gnuplot hides /usr/share/emacs/24.1/lisp/org/ob-gnuplot
/usr/share/emacs/24.1/site-lisp/org-mode/org-timer hides /usr/share/emacs/24.1/lisp/org/org-timer
/usr/share/emacs/24.1/site-lisp/org-mode/ob hides /usr/share/emacs/24.1/lisp/org/ob
/usr/share/emacs/24.1/site-lisp/org-mode/org-info hides /usr/share/emacs/24.1/lisp/org/org-info
/usr/share/emacs/24.1/site-lisp/org-mode/org-mks hides /usr/share/emacs/24.1/lisp/org/org-mks
/usr/share/emacs/24.1/site-lisp/org-mode/ob-awk hides /usr/share/emacs/24.1/lisp/org/ob-awk
/usr/share/emacs/24.1/site-lisp/org-mode/ob-perl hides /usr/share/emacs/24.1/lisp/org/ob-perl
/usr/share/emacs/24.1/site-lisp/org-mode/ob-asymptote hides /usr/share/emacs/24.1/lisp/org/ob-asymptote
/usr/share/emacs/24.1/site-lisp/org-mode/ob-sql hides /usr/share/emacs/24.1/lisp/org/ob-sql
/usr/share/emacs/24.1/site-lisp/org-mode/ob-org hides /usr/share/emacs/24.1/lisp/org/ob-org
/usr/share/emacs/24.1/site-lisp/org-mode/org-mouse hides /usr/share/emacs/24.1/lisp/org/org-mouse
/usr/share/emacs/24.1/site-lisp/org-mode/ob-haskell hides /usr/share/emacs/24.1/lisp/org/ob-haskell
/usr/share/emacs/24.1/site-lisp/org-mode/org-mhe hides /usr/share/emacs/24.1/lisp/org/org-mhe
/usr/share/emacs/24.1/site-lisp/org-mode/org-xoxo hides /usr/share/emacs/24.1/lisp/org/org-xoxo
/usr/share/emacs/24.1/site-lisp/org-mode/ob-clojure hides /usr/share/emacs/24.1/lisp/org/ob-clojure
/usr/share/emacs/24.1/site-lisp/org-mode/ob-java hides /usr/share/emacs/24.1/lisp/org/ob-java
/usr/share/emacs/24.1/site-lisp/org-mode/org-rmail hides /usr/share/emacs/24.1/lisp/org/org-rmail
/usr/share/emacs/24.1/site-lisp/org-mode/ob-emacs-lisp hides /usr/share/emacs/24.1/lisp/org/ob-emacs-lisp
/usr/share/emacs/24.1/site-lisp/org-mode/org-table hides /usr/share/emacs/24.1/lisp/org/org-table
/usr/share/emacs/24.1/site-lisp/org-mode/ob-tangle hides /usr/share/emacs/24.1/lisp/org/ob-tangle
/usr/share/emacs/24.1/site-lisp/org-mode/ob-sass hides /usr/share/emacs/24.1/lisp/org/ob-sass
/usr/share/emacs/24.1/site-lisp/org-mode/org-archive hides /usr/share/emacs/24.1/lisp/org/org-archive
/usr/share/emacs/24.1/site-lisp/org-mode/org-exp hides /usr/share/emacs/24.1/lisp/org/org-exp
/usr/share/emacs/24.1/site-lisp/org-mode/org-lparse hides /usr/share/emacs/24.1/lisp/org/org-lparse
/usr/share/emacs/24.1/site-lisp/org-mode/org-colview hides /usr/share/emacs/24.1/lisp/org/org-colview
/usr/share/emacs/24.1/site-lisp/org-mode/org-irc hides /usr/share/emacs/24.1/lisp/org/org-irc
/usr/share/emacs/24.1/site-lisp/org-mode/org-inlinetask hides /usr/share/emacs/24.1/lisp/org/org-inlinetask
/usr/share/emacs/24.1/site-lisp/org-mode/org-gnus hides /usr/share/emacs/24.1/lisp/org/org-gnus
/usr/share/emacs/24.1/site-lisp/org-mode/org-eshell hides /usr/share/emacs/24.1/lisp/org/org-eshell
/usr/share/emacs/24.1/site-lisp/org-mode/org-src hides /usr/share/emacs/24.1/lisp/org/org-src
/usr/share/emacs/24.1/site-lisp/org-mode/ob-lisp hides /usr/share/emacs/24.1/lisp/org/ob-lisp
/usr/share/emacs/24.1/site-lisp/org-mode/org-beamer hides /usr/share/emacs/24.1/lisp/org/org-beamer
/usr/share/emacs/24.1/site-lisp/org-mode/org-entities hides /usr/share/emacs/24.1/lisp/org/org-entities
/usr/share/emacs/24.1/site-lisp/org-mode/ob-latex hides /usr/share/emacs/24.1/lisp/org/ob-latex
/usr/share/emacs/24.1/site-lisp/org-mode/org-feed hides /usr/share/emacs/24.1/lisp/org/org-feed
/usr/share/emacs/24.1/site-lisp/org-mode/ob-lilypond hides /usr/share/emacs/24.1/lisp/org/ob-lilypond
/usr/share/emacs/24.1/site-lisp/org-mode/org-mew hides /usr/share/emacs/24.1/lisp/org/org-mew
/usr/share/emacs/24.1/site-lisp/org-mode/ob-octave hides /usr/share/emacs/24.1/lisp/org/ob-octave
/usr/share/emacs/24.1/site-lisp/org-mode/org hides /usr/share/emacs/24.1/lisp/org/org
/usr/share/emacs/24.1/site-lisp/org-mode/ob-table hides /usr/share/emacs/24.1/lisp/org/ob-table
/usr/share/emacs/24.1/site-lisp/org-mode/org-odt hides /usr/share/emacs/24.1/lisp/org/org-odt
/usr/share/emacs/24.1/site-lisp/org-mode/org-taskjuggler hides /usr/share/emacs/24.1/lisp/org/org-taskjuggler
/usr/share/emacs/24.1/site-lisp/org-mode/org-remember hides /usr/share/emacs/24.1/lisp/org/org-remember
/usr/share/emacs/24.1/site-lisp/org-mode/ob-scheme hides /usr/share/emacs/24.1/lisp/org/ob-scheme
/usr/share/emacs/24.1/site-lisp/org-mode/ob-maxima hides /usr/share/emacs/24.1/lisp/org/ob-maxima
/usr/share/emacs/24.1/site-lisp/org-mode/org-icalendar hides /usr/share/emacs/24.1/lisp/org/org-icalendar
/usr/share/emacs/24.1/site-lisp/org-mode/org-wl hides /usr/share/emacs/24.1/lisp/org/org-wl
/usr/share/emacs/24.1/site-lisp/org-mode/org-indent hides /usr/share/emacs/24.1/lisp/org/org-indent
/usr/share/emacs/24.1/site-lisp/org-mode/ob-R hides /usr/share/emacs/24.1/lisp/org/ob-R
/usr/share/emacs/24.1/site-lisp/org-mode/ob-C hides /usr/share/emacs/24.1/lisp/org/ob-C
/usr/share/emacs/24.1/site-lisp/org-mode/ob-screen hides /usr/share/emacs/24.1/lisp/org/ob-screen
/usr/share/emacs/24.1/site-lisp/org-mode/ob-ledger hides /usr/share/emacs/24.1/lisp/org/ob-ledger
/usr/share/emacs/24.1/site-lisp/org-mode/org-agenda hides /usr/share/emacs/24.1/lisp/org/org-agenda
/usr/share/emacs/24.1/site-lisp/org-mode/ob-python hides /usr/share/emacs/24.1/lisp/org/ob-python
/usr/share/emacs/24.1/site-lisp/org-mode/org-macs hides /usr/share/emacs/24.1/lisp/org/org-macs
/usr/share/emacs/24.1/site-lisp/org-mode/ob-js hides /usr/share/emacs/24.1/lisp/org/ob-js
/usr/share/emacs/24.1/site-lisp/org-mode/org-ctags hides /usr/share/emacs/24.1/lisp/org/org-ctags
/usr/share/emacs/24.1/site-lisp/org-mode/ob-matlab hides /usr/share/emacs/24.1/lisp/org/ob-matlab
/usr/share/emacs/24.1/site-lisp/org-mode/ob-lob hides /usr/share/emacs/24.1/lisp/org/ob-lob
/usr/share/emacs/24.1/site-lisp/org-mode/org-id hides /usr/share/emacs/24.1/lisp/org/org-id
/usr/share/emacs/24.1/site-lisp/org-mode/ob-picolisp hides /usr/share/emacs/24.1/lisp/org/ob-picolisp
/usr/share/emacs/24.1/site-lisp/org-mode/org-publish hides /usr/share/emacs/24.1/lisp/org/org-publish
/usr/share/emacs/24.1/site-lisp/org-mode/ob-css hides /usr/share/emacs/24.1/lisp/org/ob-css
/usr/share/emacs/24.1/site-lisp/org-mode/org-list hides /usr/share/emacs/24.1/lisp/org/org-list
/usr/share/emacs/24.1/site-lisp/org-mode/org-mobile hides /usr/share/emacs/24.1/lisp/org/org-mobile
/usr/share/emacs/24.1/site-lisp/org-mode/org-footnote hides /usr/share/emacs/24.1/lisp/org/org-footnote
/usr/share/emacs/24.1/site-lisp/org-mode/org-install hides /usr/share/emacs/24.1/lisp/org/org-install
/usr/share/emacs/24.1/site-lisp/org-mode/org-capture hides /usr/share/emacs/24.1/lisp/org/org-capture
/usr/share/emacs/24.1/site-lisp/org-mode/org-crypt hides /usr/share/emacs/24.1/lisp/org/org-crypt
/usr/share/emacs/24.1/site-lisp/org-mode/org-jsinfo hides /usr/share/emacs/24.1/lisp/org/org-jsinfo
/usr/share/emacs/24.1/site-lisp/org-mode/ob-plantuml hides /usr/share/emacs/24.1/lisp/org/ob-plantuml
/usr/share/emacs/24.1/site-lisp/org-mode/org-clock hides /usr/share/emacs/24.1/lisp/org/org-clock
/usr/share/emacs/24.1/site-lisp/org-mode/org-exp-blocks hides /usr/share/emacs/24.1/lisp/org/org-exp-blocks
/usr/share/emacs/24.1/site-lisp/org-mode/org-datetree hides /usr/share/emacs/24.1/lisp/org/org-datetree
/usr/share/emacs/24.1/site-lisp/org-mode/ob-ditaa hides /usr/share/emacs/24.1/lisp/org/ob-ditaa
/usr/share/emacs/24.1/site-lisp/org-mode/ob-dot hides /usr/share/emacs/24.1/lisp/org/ob-dot
/usr/share/emacs/24.1/site-lisp/org-mode/org-compat hides /usr/share/emacs/24.1/lisp/org/org-compat
/usr/share/emacs/24.1/site-lisp/org-mode/ob-exp hides /usr/share/emacs/24.1/lisp/org/ob-exp
/usr/share/emacs/24.1/site-lisp/org-mode/ob-ref hides /usr/share/emacs/24.1/lisp/org/ob-ref
/usr/share/emacs/24.1/site-lisp/org-mode/org-latex hides /usr/share/emacs/24.1/lisp/org/org-latex
/usr/share/emacs/24.1/site-lisp/org-mode/org-html hides /usr/share/emacs/24.1/lisp/org/org-html
/usr/share/emacs/24.1/site-lisp/org-mode/org-vm hides /usr/share/emacs/24.1/lisp/org/org-vm
/usr/share/emacs/24.1/site-lisp/org-mode/org-docbook hides /usr/share/emacs/24.1/lisp/org/org-docbook
/usr/share/emacs/24.1/site-lisp/org-mode/ob-ocaml hides /usr/share/emacs/24.1/lisp/org/ob-ocaml
/usr/share/emacs/24.1/site-lisp/org-mode/ob-keys hides /usr/share/emacs/24.1/lisp/org/ob-keys
/usr/share/emacs/24.1/site-lisp/org-mode/org-bibtex hides /usr/share/emacs/24.1/lisp/org/org-bibtex
/usr/share/emacs/24.1/site-lisp/org-mode/ob-comint hides /usr/share/emacs/24.1/lisp/org/ob-comint
/usr/share/emacs/24.1/site-lisp/org-mode/org-special-blocks hides /usr/share/emacs/24.1/lisp/org/org-special-blocks
/usr/share/emacs/24.1/site-lisp/flim/sasl-ntlm hides /usr/share/emacs/24.1/lisp/net/sasl-ntlm
/usr/share/emacs/24.1/site-lisp/flim/hmac-md5 hides /usr/share/emacs/24.1/lisp/net/hmac-md5
/usr/share/emacs/24.1/site-lisp/flim/ntlm hides /usr/share/emacs/24.1/lisp/net/ntlm
/usr/share/emacs/24.1/site-lisp/flim/sasl hides /usr/share/emacs/24.1/lisp/net/sasl
/usr/share/emacs/24.1/site-lisp/flim/hmac-def hides /usr/share/emacs/24.1/lisp/net/hmac-def
/usr/share/emacs/24.1/site-lisp/flim/sasl-cram hides /usr/share/emacs/24.1/lisp/net/sasl-cram
/usr/share/emacs/24.1/site-lisp/flim/sasl-digest hides /usr/share/emacs/24.1/lisp/net/sasl-digest
Features:
(shadow sort gnus-util mail-extr emacsbug message format-spec rfc822 mml
mml-sec mm-decode mm-bodies mm-encode mail-parse rfc2231 mailabbrev
gmm-utils mailheader sendmail rfc2047 rfc2045 ietf-drums mm-util
mail-prsvr mail-utils help-fns help-mode easymenu view multi-isearch
dired regexp-opt time-date tooltip ediff-hook vc-hooks lisp-float-type
mwheel x-win x-dnd tool-bar dnd fontset image fringe lisp-mode register
page menu-bar rfn-eshadow timer select scroll-bar mouse jit-lock
font-lock syntax facemenu font-core frame cham georgian utf-8-lang
misc-lang vietnamese tibetan thai tai-viet lao korean japanese hebrew
greek romanian slovak czech european ethiopic indian cyrillic chinese
case-table epa-hook jka-cmpr-hook help simple abbrev minibuffer loaddefs
button faces cus-face files text-properties overlay sha1 md5 base64
format env code-pages mule custom widget hashtable-print-readable
backquote make-network-process dbusbind dynamic-setting
system-font-setting font-render-setting move-toolbar gtk x-toolkit x
multi-tty emacs)
--
Patrick Xarles Hayes
pxhayes@gmail.com