Re: Calculations while sorting?

[Jason Stover]

On Mon, Jul 28, 2008 at 05:26:04PM +0800, John Darrington wrote:
> 1. Ben's suggestion is good idea, but I'd like to generalize upon it.
> 
>   In my local dir, I have a number of new modules in src/math with a
> similar interface to moments.c   How about we abstract this interface
> into a virtual base class (let's call it "struct statistics" for
> now).  Then instead of a "moments_reader" we can have  a polymorphic
> "statistics_reader" is possible, which takes an array of heterogenous
> "struct statistics" object.
>      
> 
> So far, I envisage:
> 
> statistics (virtual class)
> ||
> |+- linear_statistics (virtual class)
> |   ||
> |   |+- moments
> |   +-- extremes
> |
> +- order_statistics (virtual class)
>   |||
>   ||+- percentiles
>   |+-- trimmed_mean
>   +--- tukey_hinges
> 
> It gets a bit more complicated than this, because order statistics can
> only be calculated from a reader which has already been sorted,
> whereas linear statistics can be calculated from any reader.

I think this is a great idea. We should compute what we can from a single
data pass. Doing so would save many future data passes.

One thing: With the linear statistics, we could also compute the
covariance in a single data pass. I haven't added a single-pass
algorithm for it, but it isn't hard to do.