[Octave-bug-tracker] [bug #53199] hist returns different results depending on 2nd 1:n/n input argument

[Rik]

Update of bug #53199 (project octave):

                  Status:               Confirmed => Fixed                  
             Open/Closed:                    Open => Closed                 

    _______________________________________________________

Follow-up Comment #8:

I don't see how the analysis is wrong.  With every numeric calculation there
is a chance for discretization error and rounding-off to the nearest floating
point number.  In general, in a series of calculations, these are not
statistically guaranteed to cancel out.  Instead, the error tends to grow by
eps or a few eps for each calculation.

In this case, Octave has a specific Range data type which is designed for
maximum accuracy in a string of serial additions such as


range(i) = base + increment*(i-1)


Replacing the Matrix type with a Range type is enough to get Matlab-equivalent
results.


-      x = [0.5:n]'/n;
-      x = (max_val - min_val) * x  + min_val * ones (size (x));
+      ## Use range type to preserve accuracy
+      x = (0.5:n) * (1/n);
+      x = (max_val - min_val) * x + min_val;
+      x = x.';  # Convert to matrix;


This function is fairly old and hasn't received much attention over the years
so I overhauled it in this changeset
(http://hg.savannah.gnu.org/hgweb/octave/rev/1fa1869650cc).  The changeset
fixes this bug, corrects handling of non-finite values in the
auto-determination of bins, addresses problems with the validation of inputs,
and adds BIST tests for all of these issues.

For the original reporter, this version of hist will be available in the 3.4.0
release.  However, since it is just an m-file you can retrieve it from
Octave's Mercurial repository and just copy the file in to your distribution
if you want to see the changes.



    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/bugs/?53199>

_______________________________________________
  Message sent via/by Savannah
  http://savannah.gnu.org/