[Octave-bug-tracker] [bug #54572] int64 does not saturate correctly in negative direction

[Dan Sebald]

Follow-up Comment #2, bug #54572 (project octave):

Multiplication does the same:


octave:1> (intmin('int32') - 5) * 10
ans = -2147483648
octave:2> (intmin('int64') - 5) * 10
ans = 9223372036854775807


This is probably a one or two line change somewhere, but the problem is
finding where that is.  grep on INT64_MAX and variants doesn't seem to lead
anywhere.  Also, I recall JWE making all these int class arithmetic operations
consistent many years ago; so I searched the bug reports and change history
for things like int64, wrap, overflow, etc. and again that didn't lead
anywhere.

If you have the debugger working on your system, perhaps you could step into
one of these commands and follow where limit saturation is done.

    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/bugs/?54572>

_______________________________________________
  Message sent via Savannah
  https://savannah.gnu.org/