[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: Slowness in function 'open'
From: |
Robert A. Macy |
Subject: |
Re: Slowness in function 'open' |
Date: |
Thu, 21 Jun 2007 18:04:36 -0700 |
one more time...
octave 2.1.50 on WinXP 600MHz with 500MB RAM
to put the suggested header on there:
2.1.50
>> x=randn(1000,2000);
>> save -ascii "mybigfile5.txt" x;
...mybigfile5 is around 35.5MB
>> tic;load -force "mybigfile5.txt";toc*30/35.5
ans = 33.958
Wow, that really did speed up octave to give it the size of
the data set.
exit 2.1.50 and start 2.9.12...
octave-2.9.12.exe:1> tic;load("mybigfile5.txt");toc*30/35.5
same series of warnings, got before.
error: load: failed to load matrix constant
error: load: trouble reading ascii file 'mybigfile5.txt'
error: load: reading file mybigfile5.txt
ok manually removed the first # line containing info, so
the first line is
# name: x
and tried again, same list of error messages ?!
Robert
On Thu, 21 Jun 2007 20:37:19 -0400
"John W. Eaton" <address@hidden> wrote:
> On 21-Jun-2007, Mark B. wrote:
>
> | Does anybody else know how to improve these times?
>
> Maybe.
>
> As others have pointed out, if you need a quick fix for
> your problem,
> you could use a filter of some kind to convert your data
> to binary.
>
> Another option that I don't think I've seen mentioned yet
> is to filter
> your data so that it is prefixed with a header that looks
> like this:
>
> # name: x
> # type: matrix
> # rows: 100000
> # columns: 1
>
> (with the name and dimensions of your matrix substituted,
> of course).
> Then I think Octave's load function will be able to read
> the file
> faster.
>
> Or, if you are interested in improving this bit of poor
> performance so
> that everyone benefits, the functions you are looking for
> are in
> src/ls-mat-ascii.cc. The function that reads rows and
> columns of
> numbers may be trying too hard to ensure that all rows
> have the same
> number of columns and to also counting the number of
> non-comment lines
> before it actually reads the data (so it can avoid
> resizing the output
> matrix, possibly multiple times).
>
> jwe
- Re: Slowness in function 'open', (continued)
- Re: Slowness in function 'open', Dmitri A. Sergatskov, 2007/06/21
- Re: Slowness in function 'open', Robert A. Macy, 2007/06/21
- Re: Slowness in function 'open', Robert A. Macy, 2007/06/21
- Re: Slowness in function 'open', John W. Eaton, 2007/06/21
- Re: Slowness in function 'open', Robert A. Macy, 2007/06/21
Re: Slowness in function 'open', Dmitri A. Sergatskov, 2007/06/21
- Re: Slowness in function 'open', Mark B., 2007/06/21
- Re: Slowness in function 'open', Muthiah Annamalai, 2007/06/21
- Re: Slowness in function 'open', Dmitri A. Sergatskov, 2007/06/21
- Re: Slowness in function 'open', John W. Eaton, 2007/06/21
- Re: Slowness in function 'open',
Robert A. Macy <=
- Re: Slowness in function 'open', Mark B., 2007/06/22
- Message not available
- Re: Slowness in function 'open', Jordi Gutierrez Hermoso, 2007/06/22
- Re: Slowness in function 'open', John W. Eaton, 2007/06/22
Message not available
Re: Slowness in function 'open', Robert A. Macy, 2007/06/21
Re: Slowness in function 'open', Michael Creel, 2007/06/22