[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: Slowness in function 'open'
From: |
David Bateman |
Subject: |
Re: Slowness in function 'open' |
Date: |
Thu, 21 Jun 2007 22:22:23 +0200 |
User-agent: |
Thunderbird 1.5.0.7 (X11/20060921) |
Muthiah Annamalai wrote:
>
>
> On 6/21/07, *Mark B.* <address@hidden
> <mailto:address@hidden>> wrote:
>
>
> Hi,
>
> Is there a faster way to load a big text file composed only of
> integers? I
> am having severe performance penalties by using the load function
>
> These are the results of my tests:
>
> First test:
> ============================================
> Octave, version 2.9.12 (configured for "i686-pc-msdosmsvc")
>
> octave.exe:1> tic; load('mybigfile'); toc;
> Elapsed time is 98.314560 seconds.
>
> Matlab 7.3.0.267 (R2006b)
>
> >> tic; load('mybigfile');toc;
> Elapsed time is 27.923677 seconds.
>
> Octave, version 2.1.73 (i686-pc-cygwin)
>
> octave:1> tic; load('mybigfile');toc
> ans = 1117.5
> ============================================
>
> Between calls to the 'load' function I flushed the cache of the location
> where the file resides
>
> Second test:
> ============================================
> octave.exe:2> tic; load('mybigfile'); toc;
> Elapsed time is 93.594170 seconds.
>
> >> tic; load('mybigfile');toc;
> Elapsed time is 28.757878 seconds.
> ============================================
>
> (Note: I cannot save the file in binary format because of
> incompatibility
> with the C++ program that generates the text file)
>
> Is this a known issue?
>
> Thanks
> --
> View this message in context:
>
> http://www.nabble.com/Slowness-in-function-%27open%27-tf3960902.html#a11240251
> Sent from the Octave - General mailing list archive at Nabble.com
> <http://Nabble.com>.
>
> _______________________________________________
> Help-octave mailing list
> address@hidden <mailto:address@hidden>
> https://www.cae.wisc.edu/mailman/listinfo/help-octave
>
>
>
> Update your Octave version if possible. I think you should do the Octave
> developers a favor, by using a newer version of Octave 2.9.x or later.
> The comparison with Matlab, doesnt make much difference now that we are
> on a much improved version.
Muthu, If he is using 2.9.12 and MSVC build I don't think he can get any
newer without building from CVS himself :-)
>
> I would also suggest you do a offline conversion program in Octave, that
> can load the text
> files and save it to binary format using,
> save -binary cCaMeLCaPsMwItHuNgArIaNoTaTiOn
>
> and then load binary files from Octave, in your 'main' script.
>
> For me on the oldest Octave version I could find access to, 2.1.50
> a 1.4MB file loads as text in 0.399s; whereas the same file loaded as
> binary
> takes just 0.015370s.
>
> I think thats x26 improvement for me.
Yeah, binary is better... The time difference with matlab is a factor of
5 and that's not greater, but is better than cygwin... Not sure why..
However,
>
> (Note: I cannot save the file in binary format because of
> incompatibility
> with the C++ program that generates the text file)
>
> Is this a known issue?
I'd be interested in knowing what this means, as I can't decipher it..
D.