Re: [w32] display international HELLO

[Kenichi Handa]

In article <address@hidden>, "Richard Wordingham" <address@hidden> writes:

> Hindi and Malayalam are a tougher problem.  Although the basic text is 
> encoded in mule-unicode-0100-24ff, 'composition' properties are actually 
> specified in the file.  The composition property should provide renderable 
> text and mark-up which replace the basic text in the display, which ideally 
> should be totally unnecessary in an MS Windows system.  (Realising this 
> ideal requires the ability to upgrade the Uniscribe library to cover extra 
> scripts and even newly admitted characters in supported scripts.)  These 
> compositions are defined by elements for the charset indian-glyph, and its 
> characters have no specified Unicode equivalent.  You need a non-Unicode 
> font to display these characters.  Arial Unicode MS does not contain much in 
> the way of shaping tables, so it will not work properly for any of the truly 
> 'complex' Indic scripts.  (This may be why Microsoft seems to have abandoned 
> this font.)

In emacs-unicode-2 branch, I'm working on supporting Indic
(and any other scripts that require CTL (Complex Text
Layout)) by OpenType fonts (though the progress is slow).

> 2. My first discovery with Lao was that just selecting a font (Code2000) 
> that supported Lao was not enough.  It would not normally display Lao 
> characters (in the Lao charset), until I discovered that a trick such as

> (set-fontset-font "fontset-myfont" 'lao '("Code2000" . "iso10646-1"))

> suddenly made the Lao text displayable.  How does this work?  I have studied 
> the code of xdisp.c and its supporting functions, but I cannot find where 
> Emacs character codes are converted to Unicode.  I did notice that if I 
> pasted Lao in from an MS application, Emacs would accept them as Unicode 
> characters and they would be displayed properly if I selected an appropriate 
> font.

Emacs 22 still doens't unify characters in legacy charsets
and Unicode by default.  And, as Lao doesn't have official
national charset, Emacs invented one long ago (before
Unicode).  Lao characters in the HELLO file is using that
charset.  The above set-fontset-font tells Emacs to use
iso10646-1 font (i.e. a Unicode encoding font) for that Lao
charset.  Then, the CCL code ccl-encode-unicode-font
(defined in lisp/international/fontset.el) converts Lao
codes to the corresponding Unicode codes on displaying.  It
is done in x_encode_char of xterm.c (or in w32_encode_char
of w32term.c).

This ugly mechanism is not used in emacs-unicode-2.

> 3. Compositions of Lao characters, (i.e. with the 'composition' string 
> property) using the Code2000 font (the only fully working Lao font I have), 
> do not display properly, whether they are in the Lao or 
> mule-unicode-0100-24ff charset.  With the latter I have seen left-hand parts 
> of Hangul syllables displayed instead of Lao!  Perhaps when I understand how 
> uncomposed display does work, I will be able to understand this problem. At 
> present I need to defeat the composition logic by typing consonant + vowel 
> as <consonant, space, delete, vowel>!  The text entered thus then displays 
> properly, mocking the hard work that has gone into carefully composing 
> grapheme clusters.

I think it's a waste of time to learn composition mechanism
of Emacs 22.  It will be a lot improved in emacs-unicode-2.

> 4. When I explicitly specify that a buffer is to be saved in UTF-8 (or one 
> of its variants), the Lao input method suddenly switches from generating Lao 
> characters in the Lao charset to generating Lao characters in the 
> mule-unicode-0100-24ff charset.  How is this effect
> achieved?

When a buffer is created, Emacs 22 setup the char table
translation-table-for-input suitable for the buffer's
file-coding-system.  That table converts Lao charset to
mule-unicode-0100-24ff charset within an input method.

>  I can't work 
> it out.  Characters already stored in the Lao charset remain in the Lao 
> charset in the buffer, as confirmed by C-x C-e (eval-last-sexp).

> Bizarrely, selecting UTF-16 as the encoding for saving the buffer does not 
> change the charset used by the Lao charset.

Yes.  Just saving doesn't change buffer contents.  Re-read
the file.

Anyway, emacs-unicode-2 doesn't have those bizarre problems.

---
Kenichi Handa
address@hidden