bug#11073: 24.0.94; BIDI-related crash in redisplay with certain byte sequences

[Kenichi Handa]

In article <jwvvcln5ra4.fsf-monnier+INBOX@gnu.org>, Stefan Monnier 
<monnier@iro.umontreal.ca> writes:

> > Usually, yes.  But as far as there is a code space in high
> > area for a CJK charset, it is unavoidable to have a
> > buffer/string that contains a character represented by a
> > byte sequence in that high area as the test case of
> > Bug#11073.  And, as "unification" means to treat such a
> > character the same way as the unified character, I thought
> > they both have the same character code.

> Since there are two internal byte-sequence representation, I don't see
> any good reason why we shouldn't have 2 internal int representations.
> I.e. if unification failed for the byte-sequence (which might be the
> result of a bug, for all I know), we may as well keep them non-unified
> in the int representation.

Please note that not all characters in the code-space of a
CJK charset are unified.  For instance, Big5 has it's own
PUA (private use area), and characters in PUA are not
unified by default.  So, if Emacs reads a Big5 file that
contains PUA chars, those chars stay in high-area.   Then,
one can provide his own unification map that also maps PUA
chars to some Unicode chars as this:
  (unify-charset 'big5 "MyBig5.map")
After this, I thought that previously read PUA chars staying
in the high-area should be treated as the corresponding
Unicode chars (in displaying, search, etc).

One may find some bug in his map or find another map is
better.  Then he can do this again:
  (unify-charset 'big5 "MyNewBig5.map")

The current design was to enable such a scenario.

Of course, there will be an opinion that such a
functionality is too much for Emacs, and when one changes
any unification map, he must re-read a file, process-output,
mail etc.

---
Kenichi Handa
handa@m17n.org