Re: [Devel] another face->postscript.isFixedPitch :-(

[Jungshik Shin]

On Wed, 4 Jun 2003, Werner LEMBERG wrote:

Werner and Anthony,

Below is my thought.

> a longer time ago you've suggested the following:
>
> > Please let me know what you think of the attached patch to deal with
> > the halfwidth CJK fixedPitch font issue.  [...]
> >
> > Note that I have made some minor change to the patch, namely
> > increasing the threshold from 50% (">> 1") to 52% (* 0.52) because I
> > encountered a DynaComWare (DynaLab) Chinese font in which the global
> > advance width is 1000, but the ASCII characters have widths of 512.
.....
> >
> >   s p a c e d - o u t   c h a r a c t e r   p r o b l e m .   : - )
> >
> > --- freetype-2.1.3~/src/truetype/ttgload.c  Mon Sep 16 17:22:03 2002
> > +++ freetype-2.1.3/src/truetype/ttgload.c   Thu Nov 21 00:17:02 2002
> > @@ -129,7 +129,12 @@
> >      TT_Get_Metrics( &face->horizontal, idx, lsb, aw );
> >
> >      if ( check && face->postscript.isFixedPitch )
> > -      *aw = face->horizontal.advance_Width_Max;
> > +    {
> > +      if ( *aw > face->horizontal.advance_Width_Max * 0.52 || *aw == 0 )
> > +   *aw = face->horizontal.advance_Width_Max;
> > +      else
> > +   *aw = face->horizontal.advance_Width_Max >> 1;
> > +    }
> >    }
>
> Jungshik made a different proposal (which you've certainly read on
> this list), namely to do some heuristic guesses whether the
> isFixedPitch flag is correctly set.  I really don't know which
> solution should be favoured.

Just to refresh the memory, here's the recap with some comments:

js>   A. ignore monospace bit and measure individual glyphs
js>       or
js> B. to optimize a bit

js> 1. measure 10 ASCII digits and 10 Chinese characters for number 1 to
js>  10

  If 10 Chinese characters for number 1 to 10 are not available,
  use U+AC00, U+AC01,..., instead. I'll provide both lists.

js> 2. If the width of glyphs in the former group is all equal and exatly
js>  half the width of glyphs in the latter, this is a sure sign that
js>  it's a dual-width font.

    'exactly half' may need some adjustments
     according to what Anthony wrote about Dynalab fonts.

js>  3. For characters with East Asain Width Class [1] of 'N', return
js>     the width of the first group. For characters with East Asian Width
js>     class of 'W', return that of the second group.
js>     For EA W.C. of 'A'(mbiguous), it's likely to be the latter
js>     in CJK fonts, but to be sure,  measure each glyph.

   Step 3 can be omitted if including a large static array for EA
width is not desired. (Markus Kuhn has a rather compact way of getting
this info. in his implementation of wcwidth/wcswidth.) We can just measure
individual glyphs for fonts that are found to be dual width fonts in
step 2. Alternatively, step 3 can employ a very simple EA width 'table' :

   3a.  if c < 0x80 : return the width of the first group
   3b.  if c is CJK Ideograph and Hangul syllable, return that of the
        second group.
   3c.  for anything else (even though there are some known ranges
        that are certainly to have EA Width of 'W'), just measure
        individual glyphs

  Obviously, we can also use Anthony's approach combined with my
test in step 1 and 2. This has an advantage that we're not hard-coding
any width information.   Actually, I like step 1 + step 2 +
Anthony's patch along with adding IsDualWidth flag mentioned below.


>  It seems to me that the problem occurs
> with TrueType fonts only, but of course Jungshik's algorithm is
> universal and can be used with any font format.


> Another possibility is to add a new API function which simply changes
> the isFixedPitch flag to a user-defined value so that an application
> can take care of it.

   How about a new flag, isDualWidth (isDualPitch or whatever is best)?

I implemented the above heuristics in gnome-terminal, vim-gtk2, and
xterm, but every application can't be changed. In case of Xft, it
now sets the flag to ignore the global advance width'
(FT_LOAD_IGNORE_GLOBAL_ADVANCE_WIDTH) before invoking
FT2 APIs for measuring widths/extents. This solves the problem at the
cost of a small perf. loss.

  The other aspect of handling CJK dual-width fonts is that they have
to be recognized as 'monospace' fonts in some contexts. Suppose you
write a terminal emulator or a program that deals with CSS. When a user
asks for (the list of) 'monospace' fonts (in CSS), CJK dual-width
fonts  should come up because they're *semantically* monospace fonts
(although not literally). If there's a new flag 'isDualWidth',
applications can do something like the following (this is possible
now, but widths are wrong when this is set for CJK dual-width fonts)
(needless to say, this is just a pseudo-code.)

   while (*fonts_loaded++) {
     if (fonts_loaded->isFixedWidth || fonts_loaded->isDualWidth)
        mono_font_list.append(fonts_loaded);
   }

Perhaps, the test procedure described in step 1 and 2 can be done once
when a font is opened and FT_FACE_FLAG_* (dual/fixed width) flags can be
set accordingly. I realize that this can slow down the font opening. Can
we do it lazily (putting off this test as much as possible) to avoid a
performance hit?

Besides, we may add a new field advance_Width_MaxCJK' to horizontal
if the structure for face can be changed.  For dual-width fonts,
horizontal.advance_Width_Max is the width of the first group (in step
2 above) while horizontal.advance_Width_MaxCJK is the width of the
second group.


  Hope this makes sense and something similar will be implemented.

  Jungshik