[Octave-bug-tracker] [bug #63169] found At*Mr*Ec in line 37 is ~=0 my compute line 37=0

[Nicholas Jankowski]

Update of bug #63169 (project octave):

                  Status:               Need Info => Invalid                
             Open/Closed:                    Open => Closed                 

    _______________________________________________________

Follow-up Comment #2:

ok sorry, the large data block didn't render properly on my screen at first. 
I see now that you've copy/pasted the whole octave data session and results.

It seems you are concerned with the result
6.661338147750939e-16

not being exactly zero.  Octave is a numerical computation tool using floating
point arithmetic.  the numerical precision of any calculation is a complicated
affair, but it can be compared with the floating point relative accuracy,
"eps".  

>> eps
ans = 2.2204e-16

what you have in that line is on the order of zero, within limits of floating
point accuracy.  (it's about 3 eps).  while it's possible for some
calculations 3eps would represents an inordinate amount of error i don't think
that is the case here.

https://en.wikipedia.org/wiki/Floating-point_arithmetic

closing as invalid.


    _______________________________________________________

Reply to this item at:

  <https://savannah.gnu.org/bugs/?63169>

_______________________________________________
Message sent via Savannah
https://savannah.gnu.org/