[Octave-bug-tracker] [bug #63203] std and var return 0 instead of NaN for some Inf and NaN inputs

[Nicholas Jankowski]

URL:
  <https://savannah.gnu.org/bugs/?63203>

                 Summary: std and var return 0 instead of NaN for some Inf and
NaN inputs
                 Project: GNU Octave
               Submitter: nrjank
               Submitted: Wed 12 Oct 2022 03:54:58 PM EDT
                Category: Octave Function
                Severity: 3 - Normal
                Priority: 5 - Normal
              Item Group: Incorrect Result
                  Status: In Progress
             Assigned to: None
         Originator Name: Nicholas Jankowski
        Originator Email: 
             Open/Closed: Open
                 Release: dev
         Discussion Lock: Any
        Operating System: Any


    _______________________________________________________

Follow-up Comments:


-------------------------------------------------------
Date: Wed 12 Oct 2022 03:54:58 PM EDT By: Nicholas Jankowski <nrjank>
i thought for sure this had been reported before, but didn't see it.
(searching std and var are not the easiest terms to sift through.) 

var is inconsistent in values returned from Inf and NaN inputs.  Matlab
returns a NaN for all cases, but in 7.2.0 and 8.0.0:


>> var(Inf)
ans = 0

>> var([1 2 Inf])
ans = NaN

>> var([1 2 Inf]')
ans = NaN

>> var([1 2 Inf],[],1)
ans =

   0   0   0

>> var([1 2 Inf],[],2)
ans = NaN

>> var([1 2 Inf; 2 3 4],[])
ans =

   0.5000   0.5000      NaN

>> var([1 2 Inf; 2 3 Inf],[])
ans =

   0.5000   0.5000      NaN


(you get the exact same behavior with NaN instead of Inf on the inputs). 
Matlab produces a NaN in all the cases above where Octave produces a 0. 

looking through var now.  will at least add some tests/xtests to capture this
if it doesn't look like a quick fix.







    _______________________________________________________

Reply to this item at:

  <https://savannah.gnu.org/bugs/?63203>

_______________________________________________
Message sent via Savannah
https://savannah.gnu.org/