[Octave-bug-tracker] [bug #63848] Mean: some single class inputs can hit precision limit with sum

[Nicholas Jankowski]

Update of bug #63848 (project octave):

                  Status:             In Progress => Ready For Test         
         Planned Release:                    None => 9.1.0 (current default)

    _______________________________________________________

Follow-up Comment #19:

yes, turns out it was as simple as turning each call into sum (x, dim,
"double") except for the large int handling path. Pushed 
https://hg.savannah.gnu.org/hgweb/octave/rev/cfeda68b01ad 
along with BISTs to capture the single precision issue and a single xfail test
to show a similar limit with doubles. marking as ready for test


    _______________________________________________________

Reply to this item at:

  <https://savannah.gnu.org/bugs/?63848>

_______________________________________________
Message sent via Savannah
https://savannah.gnu.org/