bug#11860: 24.1; Arabic - Harakat (diacritics, short vowels) don't appear

[YAMAMOTO Mitsuharu]

>>>>> On Sun, 19 Aug 2012 13:34:36 +0900, YAMAMOTO Mitsuharu 
>>>>> <mituharu@math.s.chiba-u.ac.jp> said:

>>>>> On Sat, 18 Aug 2012 11:45:27 +0900, Kenichi Handa <handa@gnu.org> said:
>> If this problem happens only for bidi scripts, one
>> possibility is that Emacs's rendering engine (xdisp.c)
>> expects glyphs in a glyph-string are rendered in that order
>> from left to right, but the returned glyph-string on Windows
>> should be rendered in reverse order.  For instance, in the
>> above case, we may have to render glyphs in this order
>> (diacritical mark first):

>> [0 1 1593 760 0 3 6 12 4 [1 -2 0]]
>> [0 1 1593 969 8 1 8 12 4 nil]

> The font backend driver on the Mac port is supposed to support
> right-to-left shaping (including for non-BMP chars, though I don't
> have a good example), and it gives the following result (diacritical
> mark comes first) for Courier New 13pt:

>   mac-ct:-*-Courier New-normal-normal-normal-*-13-*-*-*-m-0-iso10646-1
> by these glyphs:
>   [0 1 1618 760 8 0 2 11 -8 [-1 2 1]]
>   [0 1 1593 969 8 0 6 5 4 [-1 0 8]]

The above result was not correct in a couple of points.  First, the
font backend driver for the Mac port had a bug (*1).  Second, OS X
10.7 and 10.8 seem to have a bug that they report incorrect lbearing
and rbearing values for Courier New (*2).  In particlar, the lbearing
value is always reported as 0, as in the above result.

*1: http://lists.gnu.org/archive/html/emacs-devel/2012-09/msg00157.html
*2: http://openradar.appspot.com/10377021

Mac OS X 10.6 does not have the second issue, and with the patch in
(*1), it reports the following result:

  mac-ct:-*-Courier New-normal-normal-normal-*-13-*-*-*-m-0-iso10646-1
by these glyphs:
  [0 1 1618 760 8 3 5 11 -8 [-1 2 0]]
  [0 1 1593 969 8 1 8 5 4 nil]

> In the above example, the grapheme cluster consists of glyphs having
> non-nil adjustments (the last element of each vector).  In the
> function Ffont_shape_gstring, there is some code that merges grapheme
> clusters generated by a font backend driver so each of them starts
> with a glyph having non-nil adjustment (except the first grapheme
> cluster of the gstring).  I think this is not correct especially for
> right-to-left text, and I disabled that part in the Mac port.  Could
> you give an example if you think this part is necessary?

The first glyph in the above result still has non-nil adjustments.
Another example is the Arabic "u-S-u" case for Arial 30pt (*3).  It
consists of the following two grapheme clusters (from right to left):

  [0 1 1613 755 0 1 7 2 4 [0 0 -3]]
  [0 1 0 971 16 -1 15 15 -4 nil]

  [2 2 0 970 14 1 15 13 7 [0 0 16]]

*3: http://lists.gnu.org/archive/html/bug-gnu-emacs/2012-09/msg00178.html

As you explained, the grapheme clusters are in logical order, and
glyphs in each grapheme cluster are in drawing order.  So simply
merging grapheme clusters as in the code in Ffont_shape_gstring does
not seem to be correct in the case of right-to-left text (what's drawn
later comes earlier in a merged grapheme cluster).

IMO, dividing glyphs into grapheme clusters is font backed driver's
task, and I don't understand why Ffont_shape_gstring merges the
grapheme clusters for some cases.  Could you explain?

                                     YAMAMOTO Mitsuharu
                                mituharu@math.s.chiba-u.ac.jp