Re: PSPP-BUG: moments_calculate error

[Ben Pfaff]

On Sun, Dec 30, 2012 at 09:40:43AM +0100, John Darrington wrote:
> On Wed, Dec 19, 2012 at 09:49:37PM -0800, Ben Pfaff wrote:
>      John Darrington <address@hidden> writes:
>      
>      > The pertinent assertion is:
>      >
>      >      /* After the second pass we can calculate any stat.  We
>      >          don't support "online" computation during the second
>      >          pass, so As a simple self-check, the total weight for
>      >          the passes must agree. */
>      >       assert (m->w1 == m->w2);
>      >
>      > and when I ran it through gdb, I see:
>      >
>      > Breakpoint 1, moments_calculate (m=0x81430c0, weight=0xbffff1c0, 
> mean=0xbffff1b8, variance=0xbffff1b0, skewness=0xbffff1a8, 
>      >     kurtosis=0xbffff1a0) at 
> /home/john/pspp-master/src/math/moments.c:233
>      > 233           assert (m->w1 == m->w2);
>      > (gdb) print m
>      > $2 = {max_moment = MOMENT_KURTOSIS, pass = 2, w1 = 2257.1583655975705, 
> sum = 970999467.71251988, mean = 430186.6818527166, 
>      >   w2 = 2257.1583655975778, d1 = 6.8955123424530029e-06, d2 = 
> 1504554313897766, d3 = 2.7440683550398496e+21, 
>      >   d4 = 9.0176522508330139e+27}
>      >
>      > So it would appear that this is simply a floating point
>      > precision problem (which inevitably arises when doing equality
>      > comparisons with floating points).  So I suggest that this
>      > assertion be deleted.
>      
>      w1 and w2 are each calculated as the sum of the same sequence of
>      values in the same order.  
> 
> No.  They are calculated in different orders.  The first pass iterates
> the data in the original data.  Then the data is sorted, and iterated
> for the the second pass.  It is done this way, in order to minimise
> the number of iterations through the data.

It did not occur to me that the data might be intentionally reordered
from one pass to another.  That is a valid transformation, that the
assertion does not properly account for.  If this is what's happening,
please do remove the assertion.