Re: font substitution

[Yen-Ju Chen]

On 8/12/07, Yen-Ju Chen <yjchenx@gmail.com> wrote:
> On 8/12/07, Fred Kiefer <fredkiefer@gmx.de> wrote:
> > I tried both the AR PL fonts and they seem to work for me. The problem
> > you reported in the other mail was when creating the character set, not
> > when checking if a character was included. Perhaps you could send me
> > your test file. At the moment I am not able to reproduce the problem.
> >
> > Excluding some fonts from the check wont be an option as people may want
> > to use this fonts anyway and then we have the same problem. We really
> > need to find out, what is going wrong here.
> >
> > And GNUstep should be able to support all Unicode characters, if not we
> > need to change this. From looking at the code I see no limitation.
>
>   Here is the text I used. It is in UTF-8 encoding.
>   You need a Chinese font (AR PL...)
>   and another font which has better coverage than usual,
>   probably one of the DejaVu font for a row of symbol in the bottom.

  I check the fonts again.
  The one in question is "AR PL ZenKai Uni.nfont".
  It shows a "Critical Error" of NSCharacterSet panel and
  and an exception in terminal:
  " NSImage: compositeToPoint:fromRect:operation: failed due to
    NSInternalInconsistencyException: Cannot find stored representation"

  If you want to try, there is an application in Etoile:
  Etoile/Services/User/Typewriter/
  (Well, you also need Etoile/Frameworks/OgreKit, which need oniguruma library).
  You can create a new document, choose "Edit"->"Characters...".
  It is a panel allowing you to see all of the glyphs from a chosen font.
  When you choose "ZenKai", the exception raises.
  Or you can use Etoile/Services/User/FontManager/ (no dependency).
  It has the same result.
  I have to say a bad font can be anywhere.

  Yen-Ju

>
>   Yen-J
>
> >
> > Cheers,
> > Fred
> >
> > Yen-Ju Chen wrote:
> > > On 8/11/07, Fred Kiefer <fredkiefer@gmx.de> wrote:
> > >> Thank you for the patches. I already applied the first one. For the
> > >> other two I would like to understand the problem they try to solve a bit
> > >> better, before working around it.
> > >>
> > >> The illegal value for the character set are really strange. The value
> > >> GNUstep checks  against is 1114112, which should be the maximal allowed
> > >> UNICODE value. I could understand, if GNUstep would be off by one. Could
> > >> you please test this or send me such a font, then I would have a look
> > >> myself.
> > >
> > >   I use the font here:
> > >   http://download.gna.org//etoile/etoile-default-fonts.tar.gz
> > >   One of the Chinese font (AR PL...) is broken.
> > >   Actually there are two possibility.
> > >   NSCharacterSet support longCharacterIsMember:,
> > >   which take 32-bit integer instead of unichar (16-bit).
> > >   It indicates that unicode can go beyond 16-bit.
> > >   If I use -longCharacterIsMember: to get font coverage,
> > >   it complains something is too big.
> > >   So I suspect either GNUstep cannot support 32-bit glyph,
> > >   or the font has a wrong table inside.
> > >   Actually it happens to me some time ago with another set of fonts.
> > >   Considering the nature of open source,
> > >   I am not surprised that some of the font has the wrong table inside.
> > >   For art backend, we have limited nfont packages.
> > >   So it is not a big problem to find the bad one and remove it.
> > >   But for cairo backend, it gets all fonts from X, which is a LOT.
> > >   What's the chance of any of them has broken table inside ?
> > >   So the purpose of not using all availbe fonts is
> > >   1. users can control which fonts to use and avoid bad font from system.
> > >   2. GNUstep may not support for font beyond 16-bit glyph (maybe).
> > >   And I really don't see what we can get from using all available
> > > fonts for substitution.
> > >
> > >> The matrix swapping is also strange. In many case your work around would
> > >> be fine, but in others when the matrix was explicitly set it would be
> > >> wrong. Here it would be nice to know which case are handled correct and
> > >> which wrong by the current code. Is there anything special in the way
> > >> you created you font?
> > >
> > >   The font I use is pretty much the standard font you can get (see above 
> > > link).
> > >   While I don't understand font matrix that much,
> > >   it does not make sense to me that you can apply a matrix of English font
> > >   to Chinese font, not to mention other languages, like Indian or Arabic 
> > > font.
> > >   So sharing font matrix across fonts of different languages looks
> > > more wrong to me.
> > >   And if users set matrix explicitly to the default font,
> > >   it means he want to change matrix of default font, not any other.
> > >   Automatically applying such matrix to any other fonts also seems wrong 
> > > to me.
> > >
> > >   Regards
> > >
> > >   Yen-Ju
> > >
> > >> Cheers,
> > >> Fred
> > >>
> > >> Yen-Ju Chen wrote:
> > >>> O.K. Sorry for another reply
> > >>> This patch fixes the upside-down problem.
> > >>> The attached screenshot uses English font as default font
> > >>> and Chinese font as substitute.
> > >>> I probably need to find a Japanese font and a symbol font to test
> > >>> multiple substitute.
> > >>> By the way, the speed is not bad.
> > >>> Previously I said that it takes 1-2 seconds for 25,000+ Chinese glyphs
> > >>> is including putting all glyphs on a NSMatrix.
> > >>> So it is fast to get the coverage in NSFont.
> > >>>
> > >>> Yen-Ju
> > >>>
> > >>> On 8/10/07, Yen-Ju Chen <yjchenx@gmail.com> wrote:
> > >>>> Another issue is that sometimes freetype fails to get the coverage.
> > >>>> Therefore, NSCharacterSet raise an exception:
> > >>>> "Specified range exceeds character set".
> > >>>> So this patch prevents NSAttributedString from using all available 
> > >>>> fonts
> > >>>> to eliminate the possibility of using bad fonts.
> > >>>> It only uses user specified fonts for substitution.
> > >>>> And I hope GNUstep can add an user default to specify the preferred 
> > >>>> fonts.
> > >>>>
> > >>>> Once the patch is applied, the font substitute basically works, but
> > >>>> has one problem.
> > >>>> I attach a screenshot to explain that.
> > >>>> The left window is using English font as default font and no preferred 
> > >>>> font.
> > >>>> So you only see English 'GNUstep'.
> > >>>> The middle window is using Chinese font as default font and no 
> > >>>> preferred font.
> > >>>> Because Chinese font also contains English letter,
> > >>>> you see both English and Chinese.
> > >>>> You can tell the English letter is from different font.
> > >>>> The right window is using English font as default font and Chinese
> > >>>> font as substitute.
> > >>>> Again, you can see both English and Chinese character.
> > >>>> But the funny thing is the Chinese character is upside-down.
> > >>>> I have no idea why it happens, but I suspect gui or back cache the 
> > >>>> font matrix
> > >>>> so the matrix of English font is applied to the Chinese font.
> > >>>> The hint of Chinese font is also different between middle and right 
> > >>>> window.
> > >>>> The middle one is sharp while the right one is blur.
> > >>>> It indicates something is cached, probably for the English font at the
> > >>>> start of the text.
> > >>>> Maybe someone has insights about what may cause it.
> > >>>>
> > >>>> Yen-Ju
> > >>>>
> > >>>> On 8/10/07, Yen-Ju Chen <yjchenx@gmail.com> wrote:
> > >>>>> Hi,
> > >>>>>
> > >>>>>   [NSFont coveredCharacterSet] is broken.
> > >>>>>   I attach a patch.
> > >>>>>   A quick test shows that a Chinese font which contains 25,000+ glyph
> > >>>>>   takes only about 1-2 seconds to go through.
> > >>>>>   It's on a powerbook G4 1GHz.
> > >>>>>   So cache may not be needed for coveredCharacterSet.
> > >>>>>   I haven't test the preferredFont yet.
> > >>>>>   Hope this patch can go in soon.
> > >>>>>
> > >>>>>   Yen-Ju
> > >>>>>
> > >>>>> On 8/9/07, Yen-Ju Chen <yjchenx@gmail.com> wrote:
> > >>>>>> On 8/9/07, Fred Kiefer <fredkiefer@gmx.de> wrote:
> > >>>>>>> Yen-Ju Chen wrote:
> > >>>>>>>> On 8/8/07, Fred Kiefer <fredkiefer@gmx.de> wrote:
> > >>>>>>>>> Yen-Ju Chen wrote:
> > >>>>>>>>>>   Another thing I am thinking is to save the coveredCharacterSet 
> > >>>>>>>>>> on disk.
> > >>>>>>>>>>   Font does not change frequently,
> > >>>>>>>>>>   and it does take time to generate coveredCharacterSet for each 
> > >>>>>>>>>> font,
> > >>>>>>>>>>   especially for CJK.
> > >>>>>>>>>>   If the cached coveredCharacterSet can be saved on disk and 
> > >>>>>>>>>> loaded later,
> > >>>>>>>>>>   it can reduce the overhead of starting an application.
> > >>>>>>>>>>   For art backend, it can even pack the saved 
> > >>>>>>>>>> coveredCharacterSet in nfont.
> > >>>>>>>>>>   So for small font, it is fine to check the covered character 
> > >>>>>>>>>> each time.
> > >>>>>>>>>>   For big font, saved character set is better.
> > >>>>>>>>>>
> > >>>>>>>>> Great idea! Would you like to work on that?
> > >>>>>>>>   If the patch you have goes into the -trunk,
> > >>>>>>>>   I can play with it and see how much it effects the speed
> > >>>>>>>>   and decide what's the best way to do the cache if needed.
> > >>>>>>>>
> > >>>>>>> I just committed this change, now it is your turn :-)
> > >>>>>>   Thanx. I will take a look shortly.
> > >>>>>>
> > >>>>>>   Yen-Ju
> > >>>>>>
> > >>>>
> > >>>> ------------------------------------------------------------------------
> > >>>>
> > >>>> Index: NSAttributedString.m
> > >>>> ===================================================================
> > >>>> --- NSAttributedString.m     (revision 25384)
> > >>>> +++ NSAttributedString.m     (working copy)
> > >>>> @@ -990,7 +990,7 @@
> > >>>>  - (NSFont*)_substituteFontWithName: (NSString*)fontName font: 
> > >>>> (NSFont*)baseFont
> > >>>>  {
> > >>>>    // FIXME: Catch case were baseFont is nil
> > >>>> -  return [NSFont fontWithName: fontName matrix: [baseFont matrix]];
> > >>>> +  return [NSFont fontWithName: fontName size: [baseFont pointSize]];
> > >>>>  }
> > >>>>
> > >>>>  - (NSFont*)_substituteFontFor: (unichar)uchar font: (NSFont*)baseFont 
> > >>>> fromList: (NSArray *)fonts
> > >>>> @@ -1061,13 +1061,6 @@
> > >>>>      {
> > >>>>        return subFont;
> > >>>>      }
> > >>>> -
> > >>>> -  subFont = [self _substituteFontFor: uchar font: baseFont fromList:
> > >>>> -                      [[NSFontManager sharedFontManager] 
> > >>>> availableFonts]];
> > >>>> -  if (subFont != nil)
> > >>>> -    {
> > >>>> -      return subFont;
> > >>>> -    }
> > >>>>
> > >>>>    return nil;
> > >>>>  }
> > >>>>
> > >>>> ------------------------------------------------------------------------
> > >>>>
> > >>
> > >
> >
> >
>
>