[Octave-bug-tracker] [bug #42589] Inaccurate results with the colon operator

[Michael Godfrey]

Follow-up Comment #24, bug #42589 (project octave):

@Rik,

I was afraid of that. This is the same argument that Matlab
has used. They even go so far as to not allow Matlab to
run on systems which do not provide bit-compatibility. There
was a time when this excluded a significant number of
machines. But, in any case, it reduced their numerical
accuracy.

Note that leaving extended bits on does not mean
that they need to be preserved. As Kahan points out
and his example shows, precision is improved with the
bits ignored. I have not reviewed all the current processor's
FP 754 adherence, but I suspect that practically all
current processors would yield the same results with 
extended bits on, but ignored.

You might want to look at:
http://www.cs.berkeley.edu/~wkahan/Mindless.pdf

When this is decided, it would be good to document it.



    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/bugs/?42589>

_______________________________________________
  Message sent via/by Savannah
  http://savannah.gnu.org/