[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Octave-bug-tracker] [bug #32991] format long shows more decimal digits
|
From: |
Rik |
|
Subject: |
[Octave-bug-tracker] [bug #32991] format long shows more decimal digits than can possibly be accurate |
|
Date: |
Mon, 04 Apr 2011 17:15:31 +0000 |
|
User-agent: |
Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.16) Gecko/20110323 Ubuntu/10.04 (lucid) Firefox/3.6.16 |
Follow-up Comment #2, bug #32991 (project octave):
And if you do need to be doing precise integer operations, you should consider
the integer classes (int8, uint8, int16, ...). In particular,
uint64(322426889)*uint64(404781269)
ans = 130512365289142141
You still need to consider overflow which you can do with the intmax function.
For example,
intmax('uint64')
ans = 18446744073709551615
_______________________________________________________
Reply to this item at:
<http://savannah.gnu.org/bugs/?32991>
_______________________________________________
Message sent via/by Savannah
http://savannah.gnu.org/