[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Octave-bug-tracker] [bug #30742] Function rem returns incorrect result
From: |
Jaroslav Hajek |
Subject: |
[Octave-bug-tracker] [bug #30742] Function rem returns incorrect result for certain inputs |
Date: |
Wed, 11 Aug 2010 06:26:43 +0000 |
User-agent: |
Mozilla/5.0 (X11; U; Linux x86_64; en-US) AppleWebKit/533.2 (KHTML, like Gecko) Chrome/5.0.342.9 Safari/533.2 |
Update of bug #30742 (project octave):
Status: None => Invalid
Open/Closed: Open => Closed
_______________________________________________________
Follow-up Comment #1:
No, this is a misunderstanding on your side. The default data type in Octave
is a base-2 floating point number, which can't exactly represent most decimal
numbers (the reason for base 2 is speed).
Hence, when you enter 2e-5, the closest approximation is used instead, and
likewise for 0.1e-5. That's why you don't get exact zero. Try this:
octave:1> 2e-5 / 0.1e-5 - 20
ans = 3.5527e-15
See? You can also try, e.g., in Python to confirm that this is a general
issue and nothing specific to Octave:
Python 2.6.2 (r262:71600, Jun 17 2010, 13:37:45)
[GCC 4.4.1 [gcc-4_4-branch revision 150839]] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> 2e-5 / 0.1e-5 - 20
3.5527136788005009e-15
Unless you can work this around yourself by eliminating the common negative
power of 10, you need a decimal number package. There used to be one on
OctaveForge, but currently it seems unmaintained.
_______________________________________________________
Reply to this item at:
<http://savannah.gnu.org/bugs/?30742>
_______________________________________________
Message sent via/by Savannah
http://savannah.gnu.org/