[Octave-bug-tracker] [bug #30817] truncation vs. rounding

[Lukas Reichlin]

URL:
  <http://savannah.gnu.org/bugs/?30817>

                 Summary: truncation vs. rounding
                 Project: GNU Octave
            Submitted by: paramaniac
            Submitted on: Thu 19 Aug 2010 01:12:34 PM GMT
                Category: None
                Severity: 3 - Normal
              Item Group: Incorrect Result
                  Status: None
             Assigned to: None
         Originator Name: 
        Originator Email: 
             Open/Closed: Open
         Discussion Lock: Any
                 Release: 3.2.4
        Operating System: Mac OS

    _______________________________________________________

Details:

I consider the display of result 53 as an inconsistency or a (cosmetic) bug
because round (0.5) is 1. Rounding up starts at 0.50, not at 0.51.
(Only the first digit after truncation is significant for rounding)

octave:57> 0.999996
ans =  1.00000
octave:53> 0.999995
ans =  0.99999  # this should be 1.00000 like the previous result
octave:54> 0.9999951
ans =  1.00000
octave:55> 0.9999950
ans =  0.99999  # this should be rounded up too
octave:56> 0.9999949
ans =  0.99999


Results from the "dark side":
0.99995

ans =

   1.0000  % this is correct

0.999949

ans =

   0.9999

0.999950

ans =

   1.0000

0.999951

ans =

   1.0000

0.99996

ans =

   1.0000



Cheers,
Lukas




    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/bugs/?30817>

_______________________________________________
  Message sent via/by Savannah
  http://savannah.gnu.org/