Covariance Matrix

[John Darrington]

I've been trying to hack up a reliable CORRELATIONS command.
It seemed logical to me to use the existing covariance-matrix.c in
src/math.  Whilst this worked for simple examples, it gave completely
wrong answers (or at least different from the spss examples I could 
find) when presented with either a) missing values; or b) non-unity
caseweights.

I've started digging into covariance-matrix.c in order to fix the
problems, but the exercise is slowly evolving into a complete rewrite
of the module, and in doing so, I'm not confident that I'm not 
breaking some of the functionality which correlations doesn't use 
(notably interactions).

Of course, we could just go with 2 implementations of covariance-matrix
- one which works with interactions, but not with missing values, and
the other works with missing values, but doesn't support interactions,
but I don't think this is a sensible way to proceed.

As things stand right now, I don't think there's anything in master which 
actually uses interactions, but I don't know how far Jason has got with
GLM.

So I guess the question is what is the best way to proceed?  As I see it,
the options are:

a) Fork the covariance matrix implementation and have 2 different ones
in master - not a good idea I think.

b) Get a working CM implementation which properly handles missing values 
and case-weights.   When this is working, we can add support for interactions
later.

c) Get the CM working properly with all features, including weights, missing
values and interactions, before proceeding further.


My opinion is that we should go for (b), but that's largely motivated by
personal interests, and I don't know how it's going to affect other
developers.

Comments ?


-- 
PGP Public key ID: 1024D/2DE827B3 
fingerprint = 8797 A26D 0854 2EAB 0285  A290 8A67 719C 2DE8 27B3
See http://pgp.mit.edu or any PGP keyserver for public key.

signature.asc
Description: Digital signature