Re: Compositing problem with cairo

[Doug Simons]

Thanks, Josh! You hit the nail on the head. Creating our bitmaps with the 
NSAlphaNonpremultipliedBitmapFormat flag set has fixed the problem. The only 
odd thing is that it worked on Cocoa… but maybe it just handles the case of 
alpha = 0 as a special case.

Thanks again,

Doug

On Dec 2, 2015, at 8:41 PM, Josh Freeman <gnustep_lists@twilightedge.com> wrote:

> Hi Doug,
> 
>   How are the image's alpha values being zeroed? It might be a bitmap-format 
> issue: If you're directly modifying an image's pixel data (as an 
> NSBitmapImageRep), and the bitmap's format is premultiplied-alpha 
> (([bitmapRep bitmapFormat] & NSAlphaNonpremultipliedBitmapFormat) == 0), then 
> changing a pixel's alpha value without also updating its 
> (alpha-premultiplied) RGB values would cause a compositing issue similar to 
> what you described.
> 
> Cheers,
> 
> Josh
> 
> 
> On Dec 2, 2015, at 4:31 PM, Doug Simons wrote:
> 
>> Hello,
>> 
>> I have a problem displaying an image with some transparent pixels over 
>> various colored backgrounds. We are masking out parts of an image by setting 
>> the alpha channel to 0 for some pixels, and want to allow the user to 
>> display the image over different background colors to see where the 
>> transparent pixels are. Each pixel has the alpha channel value set to either 
>> 0x0 or 0xFF.
>> 
>> Our code works perfectly on Cocoa. In GNUstep, it displays correctly over a 
>> white background. Over a black background the image appears entirely opaque, 
>> and over other colors it displays a modified version of the image color for 
>> the transparent pixels. Here are screenshots of the same image displayed 
>> over white, magenta, and black:
>> 
>> <ImageOnWhite.png><ImageOnMagenta.png><ImageOnBlack.png>
>> 
>> The image is drawn using this code in my drawRect: method:
>> 
>>    [image setCacheMode:NSImageCacheNever];
>> 
>>    [image drawAtPoint:imageDrawOrigin
>>              fromRect:imageRect
>>             operation:NSCompositeSourceOver
>>              fraction:1.0];
>> 
>> I traced the code in gdb to try to see where the NSCompositeSourceOver 
>> operator was used, but there are a lot of complexities in the code that make 
>> it hard to follow exactly what's going on. I hope maybe someone more 
>> familiar with the code can see what might be going wrong. Here is a 
>> backtrace from the call shown above down to the point where the 
>> cairo_paint() function is called.
>> 
>> (gdb) bt 6
>> #0  -[CairoGState drawGState:fromRect:toPoint:op:fraction:] (self=0x7b866e0, 
>> _cmd=0x645ceb60 <.objc_selector_list+480>,
>>    source=0xcf5eea8, aRect=..., aPoint=..., op=2, delta=1) at 
>> CairoGState.m:1463
>> #1  0x6458761a in -[GSContext(Ops) 
>> GSdraw:toPoint:fromRect:operation:fraction:] (self=0xe67c658,
>>    _cmd=0x654b92d0 <.objc_selector_list+512>, gstateNum=61, aPoint=..., 
>> srcRect=..., op=2, delta=1) at GSContext.m:858
>> #2  0x651a730d in -[NSImageRep 
>> nativeDrawInRect:fromRect:operation:fraction:] (self=0x6496c70,
>>    _cmd=0x654b9188 <.objc_selector_list+184>, dstRect=..., srcRect=..., 
>> op=2, delta=1) at NSImageRep.m:632
>> #3  0x651a964c in -[NSImageRep 
>> drawInRect:fromRect:operation:fraction:respectFlipped:hints:] 
>> (self=0x6496c70,
>>    _cmd=0x654b7bd8 <.objc_selector_list+1288>, dstRect=..., srcRect=..., 
>> op=2, delta=1, respectFlipped=0 '\000', hints=0x0)
>>    at NSImageRep.m:867
>> #4  0x65198143 in -[NSImage 
>> drawInRect:fromRect:operation:fraction:respectFlipped:hints:] 
>> (self=0x6495bd8,
>>    _cmd=0x654b7bd0 <.objc_selector_list+1280>, dstRect=..., srcRect=..., 
>> op=2, delta=1, respectFlipped=0 '\000', hints=0x0)
>>    at NSImage.m:970
>> #5  0x651979ee in -[NSImage drawAtPoint:fromRect:operation:fraction:] 
>> (self=0x6495bd8, _cmd=0xec4b38 <.objc_selector_list+56>,
>>    point=..., srcRect=..., op=2, delta=1) at NSImage.m:878
>> (More stack frames follow...)
>> (gdb)
>> 
>> This was on Windows, but we're seeing the same problem on Linux.
>> 
>> Thanks for any pointers (or better yet, code fixes!).
>> 
>> Cheers,
>> 
>> Doug
>> 
>> _______________________________________________
>> Discuss-gnustep mailing list
>> Discuss-gnustep@gnu.org
>> https://lists.gnu.org/mailman/listinfo/discuss-gnustep
>