|
From: | Rik |
Subject: | [Octave-bug-tracker] [bug #53166] format long displays 15 significant digits, rather than 15 digits after decimal point (Matlab Compatibility) |
Date: | Fri, 16 Feb 2018 12:02:03 -0500 (EST) |
User-agent: | Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:55.0) Gecko/20100101 Firefox/55.0 |
Follow-up Comment #5, bug #53166 (project octave): I asked for a test of Matlab on the Maintainer's list: Matlab: format long x = pi; y = [x; x*10; x*100; x*1000] single (y) y = 1.0e+03 * 0.003141592653590 0.031415926535898 0.314159265358979 3.141592653589793 ans = 4×1 single column vector 1.0e+03 * 0.0031416 0.0314159 0.3141593 3.1415928 In Octave, for the last value format long fixed_point_format (true) x = pi; y = [x; x*10; x*100; x*1000] y = 1.0e+03 * 0.003141592653590 0.031415926535898 0.314159265358979 3.141592653589793 When configured with exactly the same options, Matlab and Octave are equivalent. If I disable fixed_point_format the results are fixed_point_format (false) y y = 3.14159265358979 31.41592653589793 314.15926535897933 3141.59265358979292 The Matlab equivalent display for pi has one extra digit on it as mentioned earlier: Matlab format long 3.141592653589793 In general, the IEEE-854 double precision format has 15-17 significant digits (https://en.wikipedia.org/wiki/Double-precision_floating-point_format). A reasonable argument can be made that Octave should be showing the "average" case of 16 significant digits (rather than 15 which is what we do now). This would imply that we increase by 1 the number of digits we show in the "format long" case. _______________________________________________________ Reply to this item at: <http://savannah.gnu.org/bugs/?53166> _______________________________________________ Message sent via/by Savannah http://savannah.gnu.org/
[Prev in Thread] | Current Thread | [Next in Thread] |