[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: Question about Range class
From: |
Przemek Klosowski |
Subject: |
Re: Question about Range class |
Date: |
Fri, 25 Nov 2005 15:29:51 -0500 (EST) |
As an accuracy test I tried sqrt(2)*sqrt(2)-2
My TI calculator came back with... 1e-11 ...error
octave came back with... 4.44e-16 ...interesting.
You do know why this is, right? the error in such calculations is on the
order of a Least Significant Digit in whatever numbering system your machine
uses in the calculation. Octave uses double precision IEEE floating point,
which has 52 mantissa binary digits, so the error is on the order of 1/2^52
(octave variable 'eps', equal to ~ 2.220446e-16). TI calculators use a different
floating point format, BCD-based, if I remember correctly--and I'd guess it
uses 11 significant decimal digits.
-------------------------------------------------------------
Octave is freely available under the terms of the GNU GPL.
Octave's home on the web: http://www.octave.org
How to fund new projects: http://www.octave.org/funding.html
Subscription information: http://www.octave.org/archive.html
-------------------------------------------------------------
- Re: Question about Range class, (continued)
- Re: Question about Range class, Pedro Antonio Fructuoso Merino, 2005/11/23
- Re: Question about Range class, Robert A. Macy, 2005/11/23
- Re: Question about Range class, Shai Ayal, 2005/11/24
- Re: Question about Range class, John W. Eaton, 2005/11/24
- Re: Question about Range class, Joe Koski, 2005/11/24
- Re: Question about Range class, Miroslaw Kwasniak, 2005/11/25
- Re: Question about Range class, Robert A. Macy, 2005/11/25
- Re: Question about Range class,
Przemek Klosowski <=
- Re: Question about Range class, Robert A. Macy, 2005/11/25
- Re: Question about Range class, Henry F. Mollet, 2005/11/25
- Re: Question about Range class, Robert A. Macy, 2005/11/25