bug#5131: Subject: 23.1; interaction of transpose-regions with markers and multibyte chars

[schochet]

From: schochet@post.tau.ac.il
To: bug-gnu-emacs@gnu.org
Subject: 23.1; interaction of transpose-regions with markers and  
multibyte chars
--text follows this line--

Repeated use of the function transpose-regions on regions defined by
markers sometimes yields unexpected results when those regions contain
multibyte characters. In some cases the text obtained after running
transpose-regions even includes characters that were not present before.

The function reverse-all given below is designed to reverse the order
of the characters in a specified region. However, I obtain the following
results:

input region: abcd    output region: dcba               as expected
input region: ÷bcd    output region: d÷bc               expected: dcb÷
input region: ÷ab"äé  output region has CJK ideograph   expected: éä"ba÷

To reproduce this bug, simply copy to a file the text below,
beginning with the line starting with a semicolon,
visit it in emacs, and evaluate the indicated lisp expressions
by entering \C-j at the end of the indicated lines.
Note that the lisp expressions set markers to specific locations,
so the file should begin precisely where indicated.
The first character after the space after the word "case1:" should be at
position 64 in the file. If for some reason it
is not, the values given to the variable start should be adjusted.

The file below also contains an alternative function reverse-all2,
which differs from reverse-all only in using variables instead of markers.
The function reverse-all2 yields the expected results in all the above cases.

This bug does not depend on my .emacs file, since I have reproduced it with
a blank .emacs file.

Please let me know if you need any more information.

Steve Schochet

;-*- mode: lisp-interaction; coding: utf-8-unix -*-

; case 1: abcd      was: abcd
; case 2: ÷bcd      was: ÷bcd
; case 3: ÷ab"äé    was: ÷ab"äé

(progn (defvar start nil) (defvar len nil)) ;do \C-j here

; Using markers to move multi-byte characters may cause problems

(progn (setq begm (make-marker)) (setq endm (make-marker))) ;do \C-j here

(defun reverse-all ()
(set-marker begm start)
(set-marker endm (+ start (1- len)))
 (while (> endm begm)
(progn (transpose-regions begm (1+ begm) endm (1+ endm) t)
 (set-marker begm (1+ begm))
 (set-marker endm (1- endm))))) ;do \C-j here

;case1
(progn (setq start 64) (setq len 4) (reverse-all)) ;do \C-j here

;case2
(progn (setq start 94) (setq len 4) (reverse-all)) ;do \C-j here

;case3
(progn (setq start 124) (setq len 6) (reverse-all)) ;do \C-j here


; Using variables instead of markers works

(progn (defvar begv nil) (defvar endv nil))

(defun reverse-all2 ()
 (setq begv start)
 (setq endv (+ start (1- len)))
 (while (> endv begv)
(progn (transpose-regions  begv (1+ begv) endv (1+ endv) t)
 (setq begv (1+ begv))
 (setq endv (1- endv)))))

;case1
(progn (setq start 64) (setq len 4) (reverse-all2))

;case2
(progn (setq start 94) (setq len 4) (reverse-all2))

;case3
(progn (setq start 124) (setq len 6) (reverse-all2))

; end of attached file




In GNU Emacs 23.1.1 (i586-suse-linux-gnu, GTK+ Version 2.18.1)
 of 2009-10-24 on build16
Windowing system distributor `The X.Org Foundation', version 11.0.10605000
configured using `configure  '--with-pop' '--without-hesiod'  
'--with-kerberos' '--with-kerberos5' '--with-xim' '--prefix=/usr'  
'--mandir=/usr/share/man' '--infodir=/usr/share/info'  
'--datadir=/usr/share' '--localstatedir=/var'  
'--sharedstatedir=/var/lib' '--libexecdir=/usr/lib' '--with-x'  
'--with-sound' '--with-sync-input' '--with-xpm' '--with-jpeg'  
'--with-tiff' '--with-gif' '--with-png' '--with-rsvg' '--with-dbus'  
'--without-gpm' '--with-x-toolkit=gtk' '--x-includes=/usr/include'  
'--x-libraries=/usr/lib:/usr/share/X11' '--with-xft' '--with-libotf'  
'--with-m17n-flt' '--build=i586-suse-linux'  
'build_alias=i586-suse-linux' 'CC=gcc' 'CFLAGS=-fomit-frame-pointer  
-fmessage-length=0 -O2 -Wall -D_FORTIFY_SOURCE=2 -fstack-protector  
-funwind-tables -fasynchronous-unwind-tables -g -D_GNU_SOURCE  
-std=gnu89 -pipe -Wno-pointer-sign -Wno-unused-variable  
-Wno-unused-label -Wno-unprototyped-calls  
-DSYSTEM_PURESIZE_EXTRA=55000     -DSITELOAD_PURESIZE_EXTRA=10000 '  
'LDFLAGS=-Wl,-O2 -Wl,--hash-size=65521''

Important settings:
  value of $LC_ALL: nil
  value of $LC_COLLATE: nil
  value of $LC_CTYPE: nil
  value of $LC_MESSAGES: nil
  value of $LC_MONETARY: nil
  value of $LC_NUMERIC: nil
  value of $LC_TIME: nil
  value of $LANG: en_US.UTF-8
  value of $XMODIFIERS: @im=local
  locale-coding-system: utf-8-unix
  default-enable-multibyte-characters: t

Major mode: Lisp Interaction

Minor modes in effect:
  show-paren-mode: t
  tooltip-mode: t
  tool-bar-mode: t
  mouse-wheel-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  global-auto-composition-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent input:
C-x 1 <down-mouse-1> <mouse-1> C-j <down-mouse-1> <mouse-1>
C-j <down-mouse-1> <mouse-1> C-j <down-mouse-1> <mouse-1>
C-j <down-mouse-1> <mouse-1> C-j <down> <down> <down>
<down> <down> <down-mouse-1> <mouse-1> C-j C-x C-s
<up> <up> <up> <up> <up> <up> <up> <up> <up> <up> <up>
<up> <up> <up> <up> <up> <up> <up> <up> <up> <up> <up>
<up> <up> <up> <up> <up> <up> <up> <up> <up> <up> <up>
<up> <up> <up> <up> <up> <up> <up> <up> <up> <up> <up>
<help-echo> <help-echo> <help-echo> <help-echo> <help-echo>
<help-echo> <help-echo> <menu-bar> <help-menu> <se
nd-emacs-bug-report>

Recent messages:
Loading /usr/share/emacs/site-lisp/nxml-mode/rng-auto.el (source)...done
For information about GNU Emacs and the GNU system, type C-h C-a.
Invalid image size (see `max-image-size') [9 times]
Saving file /home/schochet/try/files/reverse-out.el...
Wrote /home/schochet/try/files/reverse-out.el