[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: reading text file with textscan annoyingly slow
From: |
MarcelK |
Subject: |
Re: reading text file with textscan annoyingly slow |
Date: |
Thu, 3 Nov 2011 05:28:03 -0700 (PDT) |
Well, I've installed Octave 3.4.3. and it works just fine.
Also, it works with GUIOctave if I use gnuplot as graphics toolkit.
There's even an improvement in speed when using textscan but it still takes
about 3 seconds to read one single file, which is unfortunately still too
slow for my purposes.
My intention is to read 365 of these files into a cell array,so the user
would have to wait ~20 minutes until the data is read, which is quite too
much I think.
I tried to write another function which reads the data faster but the result
is just slightly better.
Maybe somebody has some advice for me how to speed it up.
Here's the code:
function [Date1,headlines,datamatrix]=ncfread2(filename);
datamatrix=dlmread(filename,'\t',1,2);
%# read row headers
fid=fopen(filename,'r');
headerlines=fgets(fid);
index=findstr(headerlines,'}');
ncols=length(index);
headlines={};
headlines(1)=headerlines(1:index(1));
for m=2:ncols
headlines(m)=headerlines(index(m-1)+1:index(m));
endfor
%# read data as cell array of strings (need that for reading first two
columns as string)
in = fscanf(fid, '%c');
fclose(fid);
lines = regexp(in, '(^|\n)[^\n]+', 'match'); %# split to lines
data = cellfun(@(in) regexp(in, '(\b|'')[^\t]+(''|\b)', 'match'), lines,
'UniformOutput', false); %# extract fields
Date1=cell2mat(cellfun(@(x) x(1),data,'UniformOutput',false));
Date1=char(Date1(1));
%# create time vector
zeit=(cellfun(@(x) x(2),data,'UniformOutput',false)); %# time as cell array
of strings
t=zeros(size(datamatrix,1),1);
timestring=char(zeit);
for jj=1:size(timestring,1)
tstruct=strptime(timestring(jj,:),'%R');
t(jj)=tstruct.hour+tstruct.min/60;
endfor
datamatrix=[t,datamatrix];
datamatrix(:,end)=[];
endfunction
MarcelK wrote:
>
>
> I will now try to install Octave 3.4.3. and see if it works and if there's
> an improvement in speed.
>
> Marcel
>
--
View this message in context:
http://octave.1599824.n4.nabble.com/reading-text-file-with-textscan-annoyingly-slow-tp3972499p3985514.html
Sent from the Octave - General mailing list archive at Nabble.com.