Re: [PATCH] Unicode Lisp reader escapes

[Oliver Scholz]

Kenichi Handa <address@hidden> writes:

> In article <address@hidden>, Richard Stallman <address@hidden> writes:

[...]
> When you byte-compile a x.el file, x.el file is at first
> decoded.  How x.el file is decoded depends on many thing,
> and thus, of course, the resulting x.elc files become
> different.

Yes, that's what I meant.

> If you say that is a bug, I think there's no way to fix it.
>
> The very simple testcase is this:
>
> (progn
>   (let ((str "(setq x \"\300\300\")\n")
>       (coding-system-for-write 'no-conversion))
>     (write-region str nil "~/test1.el")
>     (write-region str nil "~/test2.el"))
>   (set-language-environment "Latin-1")
>   (byte-compile-file "~/test1.el")
>   (set-language-environment "Japanese")
>   (byte-compile-file "~/test2.el"))

That's not exactly what I meant. This happens basically because Emacs
has no indication on how to decode that file properly. Here's a test
case for what I had in mind:

(let ((str1 (format "\
;; -*- coding: utf-8 -*-
\(defvar my-string \"The Greek letter alpha: %c\")" (decode-char 'ucs #x3B1)))
      (str2 (format "\
;; -*- coding: iso-8859-7 -*-
\(defvar my-string \"The Greek letter alpha: %c\")" (decode-char 'ucs #x3B1))))

  (let ((coding-system-for-write 'utf-8))
    (write-region str1 nil "~/fragment-test-1.el")
    (write-region str1 nil "~/fragment-test-2.el"))

  (let ((coding-system-for-write 'iso-8859-7))
    (write-region str2 nil "~/unify-test-1.el")
    (write-region str2 nil "~/unify-test-2.el"))

  (unify-8859-on-decoding-mode -1)
  (byte-compile-file "~/unify-test-1.el") ; ch. 2913 from
                                          ; greek-iso8859-7
  (unify-8859-on-decoding-mode 1)
  (byte-compile-file "~/unify-test-2.el") ; ch. 332721 from
                                          ; mule-unicode-0100-24ff

  ;; Assuming `utf-fragment-on-decoding' is nil.
  (byte-compile-file "~/fragment-test-1.el") ; ch. 332721 from
                                             ; mule-unicode-0100-24ff

  ;; AFAICS there is no way to change the settings associated with
  ;; `utf-fragment-on-decoding' programmatically. However, the
  ;; following (taken from the variable's `defcustom' declaration)
  ;; should have the same effect as customizing it.
  (progn
    (define-translation-table 'utf-translation-table-for-decode
      utf-fragmentation-table)
    (unless (eq (get 'utf-translation-table-for-encode
                     'translation-table)
                ucs-mule-to-mule-unicode)
      (define-translation-table 'utf-translation-table-for-encode
        utf-defragmentation-table)))
  (byte-compile-file "~/fragment-test-2.el") ; ch. 2913 from
                                             ; greek-iso8859-7
  )


As Richard wrote, the fix would be to change the settings to
their default, unless the files set a specific variable.

But given the work this would require and given that the value of
changing the defaults is IMO somewhat dubious, you could as well
just document it in etc/PROBLEMS.


    Oliver
-- 
Oliver Scholz               18 Floréal an 214 de la Révolution
Ostendstr. 61               Liberté, Egalité, Fraternité!
60314 Frankfurt a. M.