[ft-devel] problems with too high BlueScale values in CFF font

[Tassyns, Bram]

Hi,

I have a CFF font with a BlueScale value of 48.
FreeType stores this value *1000 in a FT_Fixed.
Which causes an overflow resulting in a negative number.

cfftoken.h:
CFF_FIELD_FIXED_1000( 0x109, blue_scale, "BlueScale" )

As far as I can tell this value is only used in pshglob.c,
psh_blues_scale_zones to calculate 'no_overshoots'.
However because of the overflow you get a wrong value for 'no_overshoots'.
Resulting in an incorrect hinting aligment detection later in
pshglob.c, psh_blues_snap_stem. Resulting in wrong hinting.

I noticed this problem because our regression tester gives
different font parsing results between 32 and 64 bit builds.
FT_Fixed (and other as well) are not cleanly defined as
32 bit integer values (even though the documentation says signed 16.16).
Because of this, FT_Fixed is large enough to contain the real value
On a 64 bit machine, which causes it to use other hinting.

I have some ideas to fix this but would like to have a second opinion
(to prevent having my fix rejected afterwards).

1) make the blue_scale variable a double
   - this will allow both very large and very small values
   - is hardly used, so no real efficiency penalty
   - no longer need to story the value *1000
   - we could leave the old variable there for backwards compatibility
     and add a new one that has type double (ex: blue_scale_d)
   - however this introduces double into FT (as far as I know, not used
yet).

2) cut off too large values to the maximum of 16.16 when parsing
   - no changes for files that are within the allowed range for 16.16
   - result might still not be correct (just consistent between 32/64 bit)
   - one of the first operations that¹s done in the check with blue_scale
is
     multiplying it by a factor, this will have to be reworked as well.





As a separate item, it would be nice if FT_Fixed, FT_F26Dot6 and FT_F2Dot14
could be updated to be exactly 32 (resp 16) bits large so that there will
be less differences between processing a file in 32 and 64 bit.

I already tried this quickly locally using FT_Int32/FT_Int16 instead of
FT_Long/FT_Short.
However FT internally doesn't have a clear distinction between FT_Long and
FT_Fixed and
casts some FT_Long arrays to FT_Fixed arrays and vice versa. (public api
will have to change as well).
I haven't fully investigated this, but after doing these changes I get
memory corruptions when running.

Regards,
Bram