Re: Saving calculation answers to file

[Juan Pablo Carbajal]

On Tue, Jul 24, 2012 at 10:36 PM, Nicholas Jankowski
<address@hidden> wrote:
> On Tue, Jul 24, 2012 at 3:28 PM, vilsu <address@hidden> wrote:
>> Little testin:
>> sample run 3040 lines, P4 2400MHz
>>
>> Run1:
>>
>> fid = fopen ("koe.txt");
>>         for i = 1:3040
>>            x = eval (fgetl (fid))
>>            save -append tuloskoe.txt x
>>         endfor;
>> fclose (fid)
>>
>> -:took 15 s:-
>>
>>
>> Run2:
>>
>> fid = fopen ("koe.txt");
>> eof = fgetl (fid)
>> while eof != -1
>>       x = eval (eof);
>>       save -append out.txt x
>>       eof = fgetl (fid)
>> endwhile;
>> fclose (fid)
>>
>> -:took 9 s:-
>>
>>
>> Run3:
>>
>> fid = fopen ("koe.txt");
>> results = cell(3040,1);
>> for i =1:3040
>>       results{i} = eval (fgetl (fid));
>> endfor
>> save out.txt results
>> fclose (fid)
>>
>> -:took 3 s:-  !!! only 1/5 of the time of the original!!
>>
>> Thanks Juan
>>
>> vilsu
>
> Just remember, 'results' is going to get big depending on how large
> your input file gets. you had mentioned 10,000,000 lines... depending
> on the size of the ouput from each command you could run into memory
> problems by holding all of the results file at once. also the save
> -append within the loop is probably not the most efficient way to
> write the data to the file. you could use fopen before the loop, then
> some of the c-style commands to write to file one line at a time
> within the loop, then close the file after the loop. I would think it
> would be faster than 'save -append' since it wouldn't repeat the file
> handling overhead. so, if you have memory issues there are options.
> _______________________________________________
> Help-octave mailing list
> address@hidden
> https://mailman.cae.wisc.edu/listinfo/help-octave

Nicholas comments are valid points.
Also note that Run1 and Run2 are saving variables to a file with the
same name. When you try to load the file you will see only the last
result in your loop. all the other data is still in your file but you
can't load it normally in Octave. you will have to edit your file.

I insist that you go through the manual once and try to do the work
directly in Octave. You will be gaining skills with a extremely useful
language.


-- 
M. Sc. Juan Pablo Carbajal
-----
PhD Student
University of Zürich
http://ailab.ifi.uzh.ch/carbajal/