[Octave-bug-tracker] [bug #39314] convn(x, y, "valid") produces results that are different from Matlab

[Carnë Draug]

Update of bug #39314 (project octave):

                  Status:                   Fixed => Confirmed              

    _______________________________________________________

Follow-up Comment #7:

I'm reopening this bug. I thought the tests were passing but apparently I
messes up somewhere. With the change suggested we can't see the bug with
integers anymore. Comparing the results to before the patch, the fix is
correct but somewhere along the way some precision is lost (but close to
machine precision). Compare the following before and after the patch:


a = rand (10, 10, 10);
b = rand (3, 3, 3);
full  = convn (a, b, "full");
valid = convn (a, b, "valid");
d = full(3:10,3:10,3:10) - valid; # these should be exactly the same
max (d(:))
min (d(:))


I get before the patch:


octave> max (d(:))
ans =  1.4640
octave> min (d(:))
ans = -1.4087

and after the patch:


octave> max(d(:))
ans =    3.5527e-15
octave> min(d(:))
ans =   -2.6645e-15


    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/bugs/?39314>

_______________________________________________
  Message sent via/by Savannah
  http://savannah.gnu.org/