Re: Slowness in function 'open'

[Daniel J Sebald]

I've looked at the Octave code, and (barring C++ streams being
slow...I'm not familiar) I don't think there is too much that
can be done differently with the first pass loop, i.e., the 

get_lines_and_columns (is, filename, nr, nc);

routine.  (It's not like the same routine is being run twice.
The two passes are slightly different.)  Blank lines or lines
with whitespace only are ignored, so that obviates method #2,
i.e., first count the number of newline chars in binary mode.

It looks like you've programmed this to have everything checked
out in the first pass.  One could assume the first row is the
correct number of characters and rather than verify everything
in the first pass, move some tests to the second pass where it
can error if something doesn't come out right...  but I doubt
you want to do that.  So let's look at this other alternative:


> 1)  Open a temporary file in binary format and on the first pass place 
> the data into the temp file.  Reading the binary file on the second pass 
> might be faster.

I tried a "save -binary testb.m A".  In that case "load testb.m"
seems to be very quick.  So, would it be feasible to create a
temporary binary file on the first pass?  It seems that all that is
needed is to move

 d = octave_read_double (tmp_stream);

inside get_lines_and_columns()'s loop along with saving to a binary output 
stream.  Then perhaps an existing binary read method could be reused on the 
temp file.

Is it worth Jordi trying that approach?

Honestly, I don't know if it is worth the effort.  I'd just save the data in a 
binary format in the first place and leave ASCII for smaller files.

Dan