[Octave-bug-tracker] [bug #47746] imwrite writes incorrect TIFF pixel values from uint8 array on i686-linux-gnu

[Mike Miller]

Follow-up Comment #10, bug #47746 (project octave):

Ok, after some more testing (and more confusion from the oct-file to core
function conversion), I think this is now due to simple round-off error in the
values passed to GM.

It seems like if the pixel value is 256 or less, it is written as a 0. If it's
written as 257, it is written as a 1. The divisor being calculated for
quantum-depth 16 is 1/257, but round-off error is giving different quantized
pixel values on i686 vs x86_64. Should we use xround or xceil, or calculate
the divisor differently here? A quick test adding xround to the uint
TrueColorMatteType case fixes this bug.

    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/bugs/?47746>

_______________________________________________
  Message sent via/by Savannah
  http://savannah.gnu.org/