Re: [Help-gsl] Matrix multiplication precision problem ???

[Anders Aspegren Søndergaard]

On Thu, Nov 20, 2014 at 4:58 PM, Marco Dinarelli
<address@hidden> wrote:

> In order to keep track and check everything I'm doing is correct, I wrote a
> first prototype in octave and then I rewrote the same things in C with gsl.

Try without optimizations, especially without --fast-math.


> When I run the two programs, at first results are the same, but after some
> iterations to train the network, results start to diverge, with the C
> implementation having worst results and cross-entropy fluctuating values.

Your algorithm migt not be numerically stable. It could for example be chaotic.

Do you use any random number generators in your training?

> My question is: Is this a precision problem or there should be a mistake
> somewhere ?

Are you using double in both Octave and C? 1e-14 is by the way rather
close to the minimum resolvable
relative difference between two double precision numbers.

> So, in case this is a precision problem, does anybody know how to overcome
> it ?

It really depends on your algorithm. Ideally it should be robust
against small pertubations to the initial conditions,
at least if it is a neural network training algorithm (otherwise it
does not really make sense).
Try perturbing the training set/input to your octave implementation
and see if it performs about the same.
Then do that to your C implementation.


Maybe you just introduced a bug/typo/omission in your C program that
you haven't discovered yet?


Best regards
Anders A. Søndergaard