[Octave-bug-tracker] [bug #32991] format long shows more decimal digits than can possibly be accurate

[Jordi Gutiérrez Hermoso]

Update of bug #32991 (project octave):

                  Status:                    None => Confirmed              
                 Release:                   3.2.4 => dev                    
                 Summary: Incorrect result, product of bignumbers => format
long shows more decimal digits than can possibly be accurate

    _______________________________________________________

Follow-up Comment #1:

An IEEE 754 double has a 52-bit mantissa, so the largest integer that can be
represented accurately is 2^52-1, and that's approximately 15 or 16 decimal
digits. Your result is much larger than that, so it can't be represented
accurately.

The bug, if it can be deemed a bug, is that format long is displaying too many
decimal digits. It should never display more than 16. Since this is still
visible in the dev version, I'm retitling this bug and marking it for the
current dev version.

    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/bugs/?32991>

_______________________________________________
  Message sent via/by Savannah
  http://savannah.gnu.org/