bug#4736: 23.1; ns-antialias-text set to nil has no effect

[Francis Devereux]

On 17 Oct 2009, at 13:16, Adrian Robert wrote:

> http://emacsbugs.donarmstrong.com/cgi-bin/bugreport.cgi?bug=4736
> 
> I cannot reproduce this.  You must make a new frame or otherwise trigger full 
> redrawing, but it works here.  Please experiment a little, and if you can 
> narrow down the conditions that cause failure, please report back.


I can reproduce this.  If I build Emacs on Mac OS X 10.6 then ns-antialias-text 
has no effect - text is always anti-aliased (as John reports).  However, if I 
build Emacs on OS X 10.5 and then copy Emacs.app to my 10.6 machine, then 
setting ns-antialias-text to nil works as expected (i.e. it disables anti 
aliasing).

John, are you using 10.5 or 10.6?

I've also discovered that if I build Emacs as a 32 bit binary on 10.6 (the 
default is 64 bit on 10.6) with the following command then ns-antialias-text 
works.  The commands I used to build are:
LDFLAGS="-arch i386" CFLAGS="-g -O2 -arch i386 -march=core2" ../trunk/configure 
--build=i386-apple-darwin10.2.0 --with-ns && make -j2 && make install

I added some logging and built in 64 bit mode.  The output is interesting, it 
looks like the test ns_antialias_text == Qnil is returning false because 
ns_antialias_text is corrupt.  I'm not really sure what is happening though - I 
have next to no knowledge of Emacs internals.

Changes:
--- src/nsfont.m        2010-02-08 23:39:01 +0000
+++ src/nsfont.m        2010-02-14 09:07:25 +0000
@@ -1232,9 +1232,17 @@
     CGContextSetFont (gcontext, font->cgfont);
     CGContextSetFontSize (gcontext, font->size);
     if (ns_antialias_text == Qnil || font->size <= ns_antialias_threshold)
+    {
       CGContextSetShouldAntialias (gcontext, 0);
+      fprintf (stderr, "*** Disabled anti aliasing\n");
+      safe_debug_print (ns_antialias_text);
+    }
     else
+    {
       CGContextSetShouldAntialias (gcontext, 1);
+      fprintf (stderr, "*** Enabled anti aliasing, BITS_PER_EMACS_INT=%d\n", 
BITS_PER_EMACS_INT);
+      safe_debug_print (ns_antialias_text);
+    }
 
     CGContextSetTextMatrix (gcontext, fliptf);

Output from 64-bit Emacs (the following is repeated many times):
*** Enabled anti aliasing, BITS_PER_EMACS_INT=64
#<INVALID_LISP_OBJECT 0x0180006a>

Francis