[Octave-bug-tracker] [bug #49961] sign_test() returns incorrect p-value

[Kirill Pushkaryov]

Follow-up Comment #3, bug #49961 (project octave):

As far as the sign test is not a general purpose algorithm, it sounds
reasonable to leave it to the statistics package.

Unfortunately, it seems that signtest() is not entirely correct too, at least
when tail = 'right':

     switch lower(tail)
        case 'both'
          w = min(sum(x<my),sum(x>my));
          pl = binocdf(w, n, 0.5);
          p = 2*min(pl,1-pl);
        case 'left'
          w = sum(x<my);
          p = binocdf(w, n, 0.5);
        case 'right'
          w = sum(x>my);
          p = 1 - binocdf(w, n, 0.5);
        otherwise


Here we have p = 1 - binocdf(w, n, 0.5) = 1 - P(X <= w) = P(X > w), so X = w
is omitted. But p-value is by definition the probability to observe _the same_
or more extreme value under H0.

More problems I've noticed with signtest():
1) ties aren't excluded
2) incorrect results with tail = 'left':

>> [pval, h] = signtest([-ones(1, 1000) 1], 0, 'tail', 'left')
pval =  1
h = 0


We checked if a vector of a thousand -1's and one 1 has a median less than 0,
and signtest() rejected this hypothesis! In the source we have p = binocdf(w,
n, 0.5) = P(X <= sum(x < 0)). So, the more components of x are less than 0,
the closer p is to 1, but it should be the other way around.

@Kai Torben Ohlhus: Alas, I don't have any tests. I've found a paper on the
sign test (Dixon, Wilfrid J., and Alexander M. Mood. "The statistical sign
test." Journal of the American Statistical Association 41.236 (1946):
557-566), and it includes a table of critical values, but that wouldn't be
convenient for tests. I suppose, the needed values could be generated using
some trustworthy statistics software.

    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/bugs/?49961>

_______________________________________________
  Message sent via/by Savannah
  http://savannah.gnu.org/