[Octave-bug-tracker] [bug #39014] Wrong determinant for some (large) matrices

[Clemens Buchacher]

Follow-up Comment #4, bug #39014 (project octave):

Attaching det.patch with the following description:

normalize multiplication result instead of operand

The operand of a multiplication with base_det is normalized to c * 2^e,
such that 0.5 <= c < 1. The mantissa is multiplied by c, and the
exponent is updated separately. After each multiplication, the mantissa
is therefore smaller by a up to a factor 2. Despite the normalization of
the operands, the mantissa can therefore become very small. Eventually,
each multiplication suffers from severe loss of precision and the
mantissa can even get rounded to 0.

Instead, normalize the mantissa after multiplication with the operand.
This makes the operation insensitive to the number of operands.

The change may degrade accuracy in case of operands close to the maximum
or minimum representable floating point numbers, but that is not a
problem base_det tries to solve.

Bug report: https://savannah.gnu.org/bugs/?39014
Reported-by: Marco Caliari <address@hidden>
Fix-proposed-by: Stephen Montgomery-Smith <address@hidden>


(file #28118)
    _______________________________________________________

Additional Item Attachment:

File name: det.patch                      Size:1 KB


    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/bugs/?39014>

_______________________________________________
  Message sent via/by Savannah
  http://savannah.gnu.org/