[Octave-bug-tracker] [bug #39314] convn(x, y, "valid") produces results that are different from Matlab

[Carnë Draug]

Follow-up Comment #11, bug #39314 (project octave):

When I said on comment #9 "Already ahead of you :) those are already fixed" I
meant the failing tests due to wrong array dimensions mentioned on comment #8.
cset 7903ae3b657d "convn: fix incorrect tests" only fixed the tests that were
wrong.

But the bug about precision as mentioned on comment #7 still holds and that's
why the tests are failing (this is not a regression, the tests are failing
because the tests are new but the bug has always been there).

I see now that I messed up the markup on comment #7 so half of the the comment
is gone. Here is it again.

----

Compare the following before and after the patch:


a = rand (10, 10, 10);
b = rand (3, 3, 3);
full  = convn (a, b, "full");
valid = convn (a, b, "valid");
d = full(3:10,3:10,3:10) - valid; # these should be exactly the same
max (d(:))
min (d(:))


I get before the patch:


octave> max (d(:))
ans =  1.4640
octave> min (d(:))
ans = -1.4087


and after the patch:


octave> max(d(:))
ans =    3.5527e-15
octave> min(d(:))
ans =   -2.6645e-15


    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/bugs/?39314>

_______________________________________________
  Message sent via/by Savannah
  http://savannah.gnu.org/