[Octave-bug-tracker] [bug #61565] nchoosek broken for integer inputs

[Arun Giridhar]

Follow-up Comment #4, bug #61565 (project octave):

Note: Depending on the parity of k, and whether n and k are signed or
unsigned, the errors can either be about descending ranges (being fixed in bug
#61132) or a length mismatch which is caused by rounding integers differently
from floating point when divided by 2. This was exposed by recent patches to
bug #61300.


    121     ## Since Odd*Even is guaranteed to be Even, also take out a
factor
    122     ## of 2 from numerator and denominator.
    123     if (rem (k, 2))  # k is odd
    124       numer = [(v-k+1:v-(k+1)/2) .* (v-1:-1:v-(k-1)/2) / 2, v];
    125       denom = [(1:k/2) .* (k-1:-1:(k+1)/2) / 2, k];
    126     else             # k is even
    127       numer = (v-k+1:v-k/2) .* (v:-1:v-k/2+1) / 2;
    128       denom = (1:k/2) .* (k:-1:k/2+1) / 2;
    129     endif


Which means that independent of fixes to bug #61132 or bug #61300, we do need
to convert n and k to double inside nchoosek before calculating the binomial
coefficient otherwise they will round differently for float and int and cause
a length mismatch. Patch given in file #52354 does that and adds a BIST.

    _______________________________________________________

Reply to this item at:

  <https://savannah.gnu.org/bugs/?61565>

_______________________________________________
  Message sent via Savannah
  https://savannah.gnu.org/