[Octave-bug-tracker] [bug #52004] assert with tolerance

[Dan Sebald]

Follow-up Comment #1, bug #52004 (project octave):

Whenever it is one of these 1.23 != 1.23 types of error messages it is
typically a floating point precision difference between numbers.  In fact,
this assert sort of illustrates how floating point arithmetic is not the same
as real number arithmetic.

Even using long number format, the floating point granularity isn't
acknowledged.


octave:14> format long
octave:15> abs(3 - 3.1)
ans =  0.100000000000000
octave:16> abs(3 - 3.1) - 0.1
ans =    8.32667268468867e-17


The width of the answer is on the order of 10^-15, but the difference is still
smaller on the order of 10^-16, just out of range.

I'm attaching a patch for Rik's consideration since he and I worked on the
assert routine some time back.  It just adds the amount by which the tolerance
is exceeded, using only a single digit precision otherwise the message wraps
the line.  It isn't important what all the numbers are beyond the first and
the exponent.

(file #41790)
    _______________________________________________________

Additional Item Attachment:

File name: octave-assert_print_tolerance_overrun-djs2017sep12.patch Size:3 KB


    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/bugs/?52004>

_______________________________________________
  Message sent via/by Savannah
  http://savannah.gnu.org/