[Octave-bug-tracker] [bug #54572] int64 does not saturate correctly in negative direction

[John W. Eaton]

Follow-up Comment #4, bug #54572 (project octave):

Hmm, it seems to work for me:


octave:1> (intmin('int64') - 1)
ans = -9223372036854775808
octave:2> (intmin('int64') - 2)
ans = -9223372036854775808
octave:3> (intmin('int64') - 3)
ans = -9223372036854775808


What happens for you with the following?


intmin ('int64') - int64 (1)
intmin ('int64') - int64 (2)
intmin ('int64') - int64 (3)


The code for the mixed int64/double operations is in oct-inttypes.cc.  The
code for the int64 - double is just


INT_DOUBLE_BINOP_DECL (-, int64)
{
  return x + (-y);
}


in which X is an octave_int64 object and Y is a double value.  The operator +
function tries to be careful about valid ranges but is there trouble here
because the range of signed integers is not symmetric?  Or is there something
else going on here?  What code path is taken on your system?

    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/bugs/?54572>

_______________________________________________
  Message sent via Savannah
  https://savannah.gnu.org/