Re: [Gnash-commit] /srv/bzr/gnash/trunk r10784: Don't clamp color value in MovieClip.lineStyle. Fixes pollock.swf.

[Benjamin Wolsey]

>          case 2:
>          {
>              boost::uint32_t rgbval = boost::uint32_t(
> -                clamp<float>(fn.arg(1).to_number(), 0, 16777216));
> +                // See pollock.swf:
> +                // it gets too many black colors with the clamp;
> +                // it sets color to a random number from
> +                // 0 to 160000000 (about 10 times more then the max).
> +                // I guess the correct behaviour is a different
> handling
> +                // of the overflow. Letting the compiler decide on
> that
> +                // gives better results.

This is UB, and the very least bad consequence is that it's known to
give different results on different compilers. We had problems with
different matrix overflow behaviour even between 64- and 32-bit gcc due
to this. If the overflow is supposed to truncate, std::fmod is fine (or
there is a truncateWithFactor template in GnashNumeric.h, which seems
like overkill for a case like this). If it's supposed to stick to an
upper or lower limit, there is a template in Date_as.cpp (I think) that
does this safely.

bwy