emacs-orgmode
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Orgmode] adding up decimal fractions in the spreadsheet


From: Carsten Dominik
Subject: Re: [Orgmode] adding up decimal fractions in the spreadsheet
Date: Sat, 27 Sep 2008 14:19:29 +0200

Hi Rainer,

it is a common mis-conception that a number that has a limited number of digits in base 10 can be represented exactly on a computer. This is correct for integers, but not for fractional numbers.

Example:

1. Open the emacs calc with M-x calc RET
2. Type 12130.68 (first number from your example)
3. Press `p 200 RET' to switch to 200 digit precision.
4. press `d r 2 RET' o display the number in Base 2.

You'll see

10111101100010.1010111000010100011110101110000101000111101011100001010001111010111000010100011110101110000101000111101011100001010001111010111000010100011110101110000 ................


It works the other way round as well, for example 0.33333333............. has an exact representation in base 3 (0.1).

This means that even simple additions like this do produce small errors on computers, and the displayed result will depend upon how he conversion is done.

(format "%s" x) just spits out digits, as many as possible. That is why the fix I have pushed to the git repo uses

  (number-to-string res)

and not

  (format "%s" res)


number-to-string is an internal function that tries very hard to do the conversion in the way you intend it.

- Carsten

On Sep 27, 2008, at 9:19 AM, Rainer Thiel wrote:

Following a hint by Bernt Hansen, I have now changed org-table-sum to
(format "%s" res) until I'll be able to use the corrected version
Carsten Dominik uploaded to the git repository (which I am not sure
how to access).

Even so, org-table-sum at times shows a strange behaviour.

When I try to sum up the following table:

|  12130.68 |
|  -1444.19 |
| -12744.90 |
|   -186.00 |
|   7000.00 |
|  -7056.00 |
|   -335.58 |
|   -277.00 |

I get -2912.9899999999993 (-2912.99 being the exact result).  This is
something I'd expect when square roots or logarithms are implied, but
with such a simple addition and subtraction of decimal numbers I'd be
inclined to call it a bug.  Is there a way to improve org-mode in this
respect?

Many thanks again for this great mode
--
Prof. Dr. Rainer Thiel
Institut für Altertumswissenschaften
07737 Jena, Germany (EU)
address@hidden


_______________________________________________
Emacs-orgmode mailing list
Remember: use `Reply All' to send replies to the list.
address@hidden
http://lists.gnu.org/mailman/listinfo/emacs-orgmode





reply via email to

[Prev in Thread] Current Thread [Next in Thread]