variability in a test for eigs.cc

[Ben Abbott]

I occasionally see failures for eigs.cc. These failures are isolated to two 
tests, which are really the same test, but for full and sparse matrices.

>>>>> processing 
>>>>> /Users/bpabbott/Development/mercurial/local_clone/src/DLD-FUNCTIONS/eigs.cc
  26   ***** testif HAVE_ARPACK
  27  [v1,d1] = eigs(A, k, 'li');
  28  d1 = diag(d1);
  29  for i=1:k
  30   assert(max(abs((A - d1(i)*eye(n))*v1(:,i))),0.,1e-11)
  31  endfor
  32 !!!!! test failed
  33 assert (max (abs ((A - d1 (i) * eye (n)) * v1 (:, i))),0.,1e-11) expected
  34 0
  35 but got
  36  1.7641e-10
  37 maximum absolute error 1.76408e-10 exceeds tolerance 1e-11

I wrote a simple script to test the statistics for the error.

--------------
n = 20;
k = 4;
A = 
full(sparse([3:n,1:n,1:(n-2)],[1:(n-2),1:n,3:n],[1i*ones(1,n-2),4*ones(1,n),-1i*ones(1,n-2)]));

more off
err = zeros (1, 1000);
for j = 1:numel(err)
  [v1, d1] = eigs(A, k, 'li');
  d1 = diag (d1);
  for i=1:k
    err(j) = max (max (abs ((A - d1(i) * eye(n)) * v1(:,i))), err(j));
  endfor
  fprintf ("error(%2d): %9.3e\n", j, median (err(j)))
  drawnow ()
end
more on

[a, b] = hist (log10(err));
[a(:), b(:)]
--------------

The result was ...

        ans =

           773.00000   -14.58678
           154.00000   -13.80886
            28.00000   -13.03095
             9.00000   -12.25304
            10.00000   -11.47512
            10.00000   -10.69721
             9.00000    -9.91930
             6.00000    -9.14138
             0.00000    -8.36347
             1.00000    -7.58556

Which implies that a tolerance of 1e-11 can easily produce false failures.

I obtained consistent results for the sparse version.

  ***** testif HAVE_ARPACK
 [v1,d1] = eigs(A, k, 'li');
 d1 = diag(d1);
 for i=1:k
  assert(max(abs((A - d1(i)*speye(n))*v1(:,i))),0.,1e-11)
 endfor
!!!!! test failed
assert (max (abs ((A - d1 (i) * speye (n)) * v1 (:, i))),0.,1e-11) expected
0
but got
 1.9454e-11

For this case, and 1000 samples, I get ...

        ans =

           622.0000   -14.6819
           316.0000   -14.0230
            17.0000   -13.3641
            10.0000   -12.7053
            12.0000   -12.0464
             9.0000   -11.3875
             4.0000   -10.7286
             2.0000   -10.0697
             4.0000    -9.4108
             4.0000    -8.7520

I was thinking of increasing this test tolerance, but am not sure that is the 
correct thing to do.

Is this degree of variability to be expected? Does anyone else see this?

Ben