octave-maintainers
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: svds test failure


From: Daniel J Sebald
Subject: Re: svds test failure
Date: Tue, 31 Jul 2012 14:00:58 -0500
User-agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.24) Gecko/20111108 Fedora/3.1.16-1.fc14 Thunderbird/3.1.16

On 07/31/2012 01:25 PM, Daniel J Sebald wrote:
On 07/31/2012 12:47 PM, Rik wrote:
On 07/31/2012 07:02 AM, address@hidden wrote:

No, you're not. There issue is that svds has a test that is
nondeterministic and sometimes the error is higher than tolerance. One
the machine that failed, try running "test svds" a few times and see
how often that fails.

I don't know if we can make this test nondeterministic because the
randomness comes from ARPACK, not from Octave, so I don't know if we
can seed it. We could just increase the tolerance.
7/31/12

Melvin,

This issue comes up again and again. The issue is that ARPACK is an
iterative solution solver and the success or failure can depend on the
initial starting point relative to the solution. In order to stop
variability in tests, we initialize Octave's random number generators and
then create an initial seed location (which is now always the same) to
pass
to ARPACK. However, even this hasn't fully stopped all failures. It is
possible that this is due to small machine-dependent round-off errors
which
render even the initial seed location slightly different between
machines.

I agree with your assessment. However, there are six tests in svds.m and
only two of them use the 'opts' input options. The tests' fail/pass
shouldn't be dependent upon having a specific initial state for the code.

There are some other tests here that look somewhat suspect as well. For
example:

%!testif HAVE_ARPACK
%! s = svds (speye (10));
%! assert (s, ones (6, 1), 2*eps);

To expect a numerical routine to be accurate within two times the
machine precision is asking for quite a bit. The tests which have 'opts'
passed in are asking for accuracy within 1e-10, which seems more
reasonable.

I realize one would expect the result to be deterministic and not fail
in one case and pass in the other, but who knows how ARPACK is set up?

Melvin, is there a way you could make some simple mod to the svds.m
tests to indicate exactly which test(s) is failing.

I think it is the test I noted above that is causing the problem. Look at these results:

octave:4> s = svds (speye (10))
s =

   1.0000
   1.0000
   1.0000
   1.0000
   1.0000
   1.0000

octave:5> s - 1
ans =

   8.8818e-16
   6.6613e-16
   4.4409e-16
   4.4409e-16
   2.2204e-16
   0.0000e+00

octave:6> eps
ans =  2.2204e-16
octave:7> assert (s, ones (6, 1), 2*eps)
error: assert (s,ones (6, 1),2 * eps) expected
   1
   1
   1
   1
   1
   1
but got
   1.0000
   1.0000
   1.0000
   1.0000
   1.0000
   1.0000
maximum absolute error 8.88178e-16 exceeds tolerance 4.44089e-16
error: called from:
error: /usr/local/src/octave/octave-builtin_bin2dec/octave-build1/../octave/scripts/testfun/assert.m at line 235, column 5
octave:7> s = svds (speye (10))
s =

   1.0000
   1.0000
   1.0000
   1.0000
   1.0000
   1.0000

octave:8> s - 1
ans =

   6.6613e-16
   4.4409e-16
   2.2204e-16
   0.0000e+00
   0.0000e+00
   0.0000e+00

octave:9>

Two things to note above. One is that the result isn't accurate to within 2*eps and the test fails. The second is that the results are *not* deterministic. Therefore, it is random that this test could pass or fail. I suggest bumping the tolerance up to 10*eps or maybe 100*eps... or 1e-10 if you like, same as the rest. It is simply trying to test the results are fairly accurate, right?

There is one test here that has no tolerance, expecting exact accuracy.

%!testif HAVE_ARPACK
%! [u2,s2,v2,flag] = svds (zeros (10), k);
%! assert (u2, eye (10, k));
%! assert (s2, zeros (k));
%! assert (v2, eye (10, 7));

octave:10> [u2,s2,v2,flag] = svds (zeros (10), k);
octave:12> u2
u2 =

Diagonal Matrix

   1   0   0   0   0   0   0
   0   1   0   0   0   0   0
   0   0   1   0   0   0   0
   0   0   0   1   0   0   0
   0   0   0   0   1   0   0
   0   0   0   0   0   1   0
   0   0   0   0   0   0   1
   0   0   0   0   0   0   0
   0   0   0   0   0   0   0
   0   0   0   0   0   0   0

octave:13> u2 - eye (10,k)
ans =

Diagonal Matrix

   0   0   0   0   0   0   0
   0   0   0   0   0   0   0
   0   0   0   0   0   0   0
   0   0   0   0   0   0   0
   0   0   0   0   0   0   0
   0   0   0   0   0   0   0
   0   0   0   0   0   0   0
   0   0   0   0   0   0   0
   0   0   0   0   0   0   0
   0   0   0   0   0   0   0

octave:14> s2
s2 =

Diagonal Matrix

   0   0   0   0   0   0   0
   0   0   0   0   0   0   0
   0   0   0   0   0   0   0
   0   0   0   0   0   0   0
   0   0   0   0   0   0   0
   0   0   0   0   0   0   0
   0   0   0   0   0   0   0

octave:15> s2 - zeros (k)
ans =

   0  -0  -0  -0  -0  -0  -0
  -0   0  -0  -0  -0  -0  -0
  -0  -0   0  -0  -0  -0  -0
  -0  -0  -0   0  -0  -0  -0
  -0  -0  -0  -0   0  -0  -0
  -0  -0  -0  -0  -0   0  -0
  -0  -0  -0  -0  -0  -0   0

octave:16> v2
v2 =

Diagonal Matrix

   1   0   0   0   0   0   0
   0   1   0   0   0   0   0
   0   0   1   0   0   0   0
   0   0   0   1   0   0   0
   0   0   0   0   1   0   0
   0   0   0   0   0   1   0
   0   0   0   0   0   0   1
   0   0   0   0   0   0   0
   0   0   0   0   0   0   0
   0   0   0   0   0   0   0

octave:18> v2 - eye (10, 7)
ans =

Diagonal Matrix

   0   0   0   0   0   0   0
   0   0   0   0   0   0   0
   0   0   0   0   0   0   0
   0   0   0   0   0   0   0
   0   0   0   0   0   0   0
   0   0   0   0   0   0   0
   0   0   0   0   0   0   0
   0   0   0   0   0   0   0
   0   0   0   0   0   0   0
   0   0   0   0   0   0   0

The reason this works out I suspect is because ARPACK's internal routine does no iterations for this case, seeing as everything is zeros. That, or minimal computations because for some odd reason the 's2' check above shows all negative zero except for positive zero on the diagonal.

Dan


reply via email to

[Prev in Thread] Current Thread [Next in Thread]