[Octave-bug-tracker] [bug #47469] sizemax() is off by one

[Carnë Draug]

URL:
  <http://savannah.gnu.org/bugs/?47469>

                 Summary: sizemax() is off by one
                 Project: GNU Octave
            Submitted by: carandraug
            Submitted on: Sat 19 Mar 2016 19:07:59 GMT
                Category: Octave Function
                Severity: 3 - Normal
                Priority: 5 - Normal
              Item Group: Incorrect Result
                  Status: None
             Assigned to: None
         Originator Name: 
        Originator Email: 
             Open/Closed: Open
         Discussion Lock: Any
                 Release: dev
        Operating System: Any

    _______________________________________________________

Details:

The function sizemax() is off by one.


octave> sizemax ()
ans = 2147483646
octave> intmax ("int32")
ans = 2147483647
octave> uint64 (numel (ones (1, 2147483647)))
ans = 2147483647


The fact that I can create that array of ones is a separate bug (see bug
#47468). The reason for being off by one is mentioned in the source of
dim-vector:

   
http://hg.savannah.gnu.org/hgweb/octave/file/2cdbae31c022/liboctave/array/dim-vector.cc#l32

Checking the logs for a better explanation brings up this:

    http://hg.savannah.gnu.org/hgweb/octave/rev/6683f0c9d742

I removed the "-1" from dim_vector::dim_max and make check reports no new
failures. Maybe it is no longer needed (or maybe it's some untested codepath.
Anyone has any insight?




    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/bugs/?47469>

_______________________________________________
  Message sent via/by Savannah
  http://savannah.gnu.org/