[Octave-bug-tracker] [bug #53166] format long displays 15 significant digits, rather than 15 digits after decimal point (Matlab Compatibility)

[Rik]

Follow-up Comment #5, bug #53166 (project octave):

I asked for a test of Matlab on the Maintainer's list:

Matlab:


 format long
 x = pi;
 y = [x; x*10; x*100; x*1000]
 single (y)

y =

   1.0e+03 *

   0.003141592653590
   0.031415926535898
   0.314159265358979
   3.141592653589793


ans =

  4×1 single column vector

   1.0e+03 *

   0.0031416
   0.0314159
   0.3141593
   3.1415928


In Octave, for the last value


format long
fixed_point_format (true)
x = pi;
y = [x; x*10; x*100; x*1000]
y =

  1.0e+03 *

   0.003141592653590
   0.031415926535898
   0.314159265358979
   3.141592653589793



When configured with exactly the same options, Matlab and Octave are
equivalent.  If I disable fixed_point_format the results are


fixed_point_format (false)
y
y =

      3.14159265358979
     31.41592653589793
    314.15926535897933
   3141.59265358979292



The Matlab equivalent display for pi has one extra digit on it as mentioned
earlier:

Matlab


format long
3.141592653589793


In general, the IEEE-854 double precision format has 15-17 significant digits
(https://en.wikipedia.org/wiki/Double-precision_floating-point_format).  A
reasonable argument can be made that Octave should be showing the "average"
case of 16 significant digits (rather than 15 which is what we do now).  This
would imply that we increase by 1 the number of digits we show in the "format
long" case.




    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/bugs/?53166>

_______________________________________________
  Message sent via/by Savannah
  http://savannah.gnu.org/