[Octave-bug-tracker] [bug #54572] int64 does not saturate correctly in negative direction

[Dan Sebald]

Follow-up Comment #9, bug #54572 (project octave):


octave:4> __octave_config_info__ ("CC")
ans = gcc
octave:5> exit
@linux ~/ $ gcc --version
gcc (Ubuntu 5.4.0-6ubuntu1~16.04.10) 5.4.0 20160609


I wonder if this is just that floating point 53-bit mantissa (significand)
issue when converting to int64:


octave:14> int64(-9223372036854775808)
ans = -9223372036854775808
octave:15> int64(-9223372036854775807)
ans = -9223372036854775808
octave:16> int64(-9223372036854775806)
ans = -9223372036854775808


Then again, maybe not:


octave:45> x = intmin("int64")
x = -9223372036854775808
octave:46> --x
ans = 9223372036854775807
octave:47> class(x)
ans = int64


    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/bugs/?54572>

_______________________________________________
  Message sent via Savannah
  https://savannah.gnu.org/