Re: Why is sscanf() so slow?

[stefan]

On Wed, 26 Mar 2003, Paul Kienzle wrote:

> >>| first of all:  Thanks for great software.  I am using it now for about two
> >>| years, especialy for displaying and adjusting measured data (comes from
> >>| some measurement bus system to computer).  The software which drives these
> >>| devices almostly always produce tab- or comma-seperated data.
> >>|
> >>| For this I do:
> >>|
> >>|     while (isstr(line = fgets(fid)))
> >>|       if (index(line, "#") == 1)
> >>|         continue;
> >>|       endif
> >>|       if (length(line) <= 2)
> >>|         continue;
> >>|       endif
> >>|       numbers = sscanf(line, "%g %g %g %g %g %g %g %g", 8);
> >>|       for i = 1 : length(numbers)
> >>|         values(n, i) = numbers(i);
> >>|       endfor
> >>|       n = n + 1;
> >>|     endwhile
> >>|
> >>| or very likely...
> >>|
> >>| For more than 1000 lines this takes *ages*.
> >>
> >>Try preallocating the values array.  Assuming you haven't done that,
> >>then the code above resized values many times.  If you are not sure of
> >>the final size, allocate something large, reallocating if necessary
> >>(but in large chunks, perhaps 2x previous allocation, or similary),
> >>then resize when you reach eof.  Sscanf is also slow.  If you just
> >>have numbers in the file, maybe you want to use load instead.  With
> >>current snapshots, it should support command and tab delimited files
> >>with comments (beginning with # or % characters, I think).
> >>
> >>
> >
> >Right now I am using a modified version of the 'aload.m' which some on
> >this mailing list sent to me.
> >
> >I'll also try your hint and tell you whether it gets better...

I investigated the problem some more.  I tried using preallocated arrays
with no harm to the time consumed.  Thus I started commenting every line
in the 'while' loop and finally found the bogus (*very* slow) function:

When replacing

      if (index(line, "#") == 1)
        continue;
      endif

by

      if (line(1) == '#')
        continue;
      endif

it works much, much, much faster.  Don't know why, but it is.  'sscanf()'
wasn't the limiting function.

Cheers,
        address@hidden



-------------------------------------------------------------
Octave is freely available under the terms of the GNU GPL.

Octave's home on the web:  http://www.octave.org
How to fund new projects:  http://www.octave.org/funding.html
Subscription information:  http://www.octave.org/archive.html
-------------------------------------------------------------