ftfont ISO10646-1 font bug found (was Re: 23.0.60; Heavy display problems with new font backend)

[sand]

address@hidden writes:
> David Hansen writes:
> I traced the fonts being sent to ftfont_has_char() and discovered that
> for a letter like LATIN SMALL LETTER S ACUTE (#x15b), Emacs was asking
> about the Neep Alt family as expected, but only about fonts with the
> iso8859-1 registry/encoding within that family.  When I removed the
> iso8859-1 fonts entirely from the font path and HUPed the font server,
> Emacs started asking about the iso10646-1 registry/encoding fonts, and
> those *were* found and displayed correctly.  (On the other hand, the
> mode line's file name---and just the mode line's file name---dropped
> down to a different font.)  So:
> 
>  1. It looks like Emacs has a problem determining the right registry
>  to use for certain code points, or the font picking fallback code has
>  problems.
> 
>  2. The file name display code may not be correctly integrated with
>  the new font backend.
> 
> At this point I think I need to get more familiar with the contents of
> etc/DEBUG...

I think I have found the cause for problem #1.  In ftfont_list(), the
code gathers a list of candidate fonts that match the
foundry/family/... requirements:

    objset = FcObjectSetBuild (FC_FOUNDRY, FC_FAMILY, FC_WEIGHT, FC_SLANT,
                   FC_WIDTH, FC_PIXEL_SIZE, FC_SPACING,
                   FC_CHARSET, FC_FILE,
  #ifdef FC_FONTFORMAT
                   FC_FONTFORMAT,
  #endif    /* FC_FONTFORMAT */
                   NULL);
  /* ... elided ... */
  fontset = FcFontList (NULL, pattern, objset);

Note that this doesn't include any registry restriction.

The code loops across the returned fontsets, calling
ftfont_pattern_entity() to generate font_entity structs.  But at no
point does it attempt to filter the font list by compatible
registries.  We get, for example:

(gdb) frame
#0  ftfont_pattern_entity (p=0x89fce40, frame=148009620, registry=138791553) at 
/home/upham/src/emacs/Apollo/emacs-cvs/src/ftfont.c:116
(gdb) p file
$151 = (FcChar8 *) 0x8a70688 
"/home/upham/.fonts/jmk/neep-alt-iso8859-1-06x11.pcf.gz"
(gdb) p registry
$152 = 138791553
(gdb) xpr registry
Lisp_Symbol
$153 = (struct Lisp_Symbol *) 0x845ca80
"iso10646-1"

Emacs will think that "neep-alt-iso8859-1-06x11.pcf.gz" is a valid
font for displaying "iso10646-1", but it isn't, and we end up with
missing code points.

This explains why removing the iso8859-1 fonts fixed the problem
(except for the mode line file name): the current code also points
iso8859-1 requesters to iso10646-1 fonts, and those always work.  I
also think this explains why I don't see this consistently across
hosts: depending on how the font list is ordered (maybe due to inode
ordering on disk?), some hosts will get a correct iso10646-1 ->
iso10646-1 mapping first at display time, while others will get an
incorrect iso10646-1 -> iso8859-1 mapping.

Another family that should have the same problem is misc-fixed, as it
also has both iso8859-1 and iso10646-1 registry fonts.  There may be
other families that I'm not aware of.

Derek

-- 
Derek Upham
address@hidden