[Octave-bug-tracker] [bug #43369] rats has some internal overflow

[Colin Macdonald]

URL:
  <http://savannah.gnu.org/bugs/?43369>

                 Summary: rats has some internal overflow
                 Project: GNU Octave
            Submitted by: cbm
            Submitted on: Mon 06 Oct 2014 11:08:35 PM GMT
                Category: Octave Function
                Severity: 3 - Normal
                Priority: 5 - Normal
              Item Group: Incorrect Result
                  Status: None
             Assigned to: None
         Originator Name: 
        Originator Email: 
             Open/Closed: Open
         Discussion Lock: Any
                 Release: dev
        Operating System: Any

    _______________________________________________________

Details:

Consider:

octave:33> rats(2.005, 25)
ans =                   401/200
octave:34> rats(2.0005, 25)
ans =   -2147483648/-2147483648


Matlab seems ok.  "rats(pi, N)" seems to max out at "80143857/25510582" (with
error 2*eps) at around N=30 but still produces that string with additional
padding for N=1000.





    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/bugs/?43369>

_______________________________________________
  Message sent via/by Savannah
  http://savannah.gnu.org/