Re: autocov and autocor

[Michael Godfrey]

On 10/28/2010 11:30 AM, John W. Eaton wrote:

On 28-Oct-2010, Rik wrote:

| I'd done some checking, but hadn't consolidated everything in my mind to
| write up.  The short answer is that there is very little definitional
| certainty about how to calculate the cross-covariance (and hence the
| special case of the autocovariance).

OK.

| Although the bug poster was
| *emphatic* about the correct way to do things, in fact, it varies between
| academic fields and even when the same method is used there is a
| normalization factor which varies.

Yeah, well at this point, he's changed his opinion a bit.  See his
latest update on the bug tracker.

| In this case, where there is no clear cut standard, my leaning is to follow
| Matlab; at least we can all be wrong together.

Yes, at least then programs will work the same way in both, which is
what I think most users expect.

| Still, I think overall
| these functions are too esoteric and should be removed from the
| core.

I'm OK with moving these functions and perhaps others that are not in
the Matlab core and that are not widely used to an external package.

Likewise, I think it migth also be a good idea to move most of the
functions in scripts/statistics to the Octave Forge statistics
package.

jwe

John,

This seems reasonable.  The arguments about the "right" way to compute these
functions are misguided.  This is a statistical question and quite a lot of thought
(including my own) has gone into these issues.  Among other things (like
numerical stability) the best choice depends on the data and on what you hope
to learn about it.

The matlab folks worried about this briefly quite a while ago and ended up
deciding that learning from statisticians was too much trouble.  Their current
code is not good enough.

So, the brief summary is that these things should be left to people who know
how to do the work and why they are doing it.  R has good statistical functions,
written by numerical statisticians.

Michael