Re: integer arithmetics

[Jaroslav Hajek]

On Thu, Oct 2, 2008 at 9:07 PM, John W. Eaton <address@hidden> wrote:
> On  2-Oct-2008, Jaroslav Hajek wrote:
>
> | > Do you mean this comment?
> | >
> | > | +  // XXX: This trick is to not lose precision for things like 3*2**63 
> + intmin('int64')
> | >
> | > Can you elaborate?
> |
> | OK, here's a detailed problem description: The problem is that
> | convertnig 64-bit integers to doubles is a lossy operation. If 64-bit
> | ints were handled exactly like 32-bit and smaller, then, for instance,
> | 4611686018427387901 (int64) + (double) 1 would produce
> | 4611686018427387904 = 2**62
> | Though this may be easily considered "a feature", it would cause
> | expressions like
> | a = a + 2 to produce unexpected results.
> |
> | If we have at least 80 bits wide long double (gcc on x86 archs), the
> | problem is easily solved, as all 64-bit integers fit into long double
> | without losing precision.
> | If not, we're in trouble.
> | Addition and subtraction are still relatively easy to handle. When
> | doing x + y (x is octave_int64, y is double), I first check whether y
> | is within the range of int64. If yes, then x + octave_int64(y)
> | produces exactly the same results as the long double version would.
> | If not, the situation is more complicated, because, for instance,
> | expression like 3*2**62 + (1+intmin('int64')) should produce 2**62 +
> | 1.
> | In other words, if the double operand is within twice the signed
> | integer range, the result may still not overflow. Here I use the fact
> | that if y is outside the signed int64 range, it must be divisible by 2
> | without going fractional. So I split it into two halfs (each of which
> | must be within int64 range or the result would overflow anyway) and do
> | two converts and two additions. The situation is slightly different
> | for unsigned integers (no /2 trick) but the code should also work
> | always.
>
> OK, maybe some of this should go in the comments in the code, so
> people reading it later won't have to rediscover it?
>

I did put some comments there. If you think there should be more,
please suggest what should be added.

> jwe
>



-- 
RNDr. Jaroslav Hajek
computing expert
Aeronautical Research and Test Institute (VZLU)
Prague, Czech Republic
url: www.highegg.matfyz.cz