Re: [Discuss-gnuradio] [VOLK] 16i_max_star_horizontal_16i non-saturating subtraction

[Tom Tsou]

On Thu, Feb 6, 2014 at 11:58 AM, Douglas Geiger
<address@hidden> wrote:
> I'll suggest that the generic kernel should be trusted to do the correct
> thing, or at least have the correct set of intentions, and that if you want
> to change the behavior of the generic kernel, that you are introducing a
> change that will break applications depending on this behavior.
>
> I believe the use of casting the subtraction before the comparison (where:
> ((int16_t) (src0[i] - src0[i + 1]) > 0) ? src0[i] : src0[i+1]; is the full
> comparison) was intentional to ensure overflow subtraction vs. saturation
> subtraction.

Assuming this kernel is used for some type of trellis decoding, the
overflow makes sense as it avoids the need for normalization of
accumulated metrics. It does, however, require that the spread of
input values be bounded for it to work. Otherwise, you have
comparisons between overflow values and non-overflow values, which
makes the comparison ambiguous and the output not-sane.

  -TT