Re: Improving strread / textread / textscan

[Philip Nienhuis]

Ben Abbott wrote:
> On Oct 23, 2011, at 8:37 PM, Ben Abbott wrote:
>
> > I'll prepare a changeset.
> >
> > Ben
>
> A changeset is atached (I won't push until I get feedback)
>
> It appears to me that whitespace is treated different from delimiters. Specifically repeated 
> whitespace is always treated as a single delimiter (i'll need to check what happens when the 
> "delimiter" parameter include white-space characters), and repeated delimiters imply an 
> empty field which (for numeric data) is set equal to "emptyvalue".

Just as a., b., c. and d. in my original posting, isn't it?

> Thus, "emptyvalue" only substituted for missing numeric data between non-white space 
> delimiters (those characters explicitly specified as "delimiters")

Again, just as d.

> The default value for "emptyvalue" is NaN, unless NaN cannot be represented by 
> the data type (int32 for example). When NaN isn't value zero is used.

Yes, cf. ML docs. This looks consistent to me.

> I've added some tests and xtests. Some of the xtests conflict with tests. I've 
> added comments to the tests in the hope of avoiding confusion.
>
> I still plan to add more tests for other functionality which is present in ML 
> but not in Octave. I'll add those as xtests as well and push them as I don't 
> expect any discussion is needed.

Well, perhaps we'd better wait until this is all sorted out.

I've already patched strread / textscan / textread on my dev machine 
(esp. strread as that does the work for textscan) for "proper" 
whitespace/delimiter handling, + better code (Rik rightly complained 
about too "complicated" parts).
Some of those changes overlap with your patches.

BTW, there IS an emptyvalue texinfo string, in strread. Octave's 
textscan.m refers to strread.m for all parameters not exclusive to 
textscan.m (same goes for textread).
(What's lacking is mentioning the default value of NaN, except for int32.)
So I think the texinfo string in strread.m should be amended, rather 
than adding one in textscan.m

Thank you for your efforts here! I have only patchy access to ML 
currently, so your sorting out is a big help.


Philip