Re: Implementation of importdata belonging to Octave Core or Octave-Forge?

[Philip Nienhuis]

Juan Pablo Carbajal wrote:
> On Sat, Oct 20, 2012 at 4:49 PM, Philip Nienhuis
> <address@hidden>  wrote:
> > Erik Kjellson wrote
> > > On 19 October 2012 15:54, Jordi Gutiérrez Hermoso&lt;
> >
> > > jordigh@
> >
> > > &gt;wrote:
> > >
> > > > On 19 October 2012 09:37, Juan Pablo Carbajal&lt;
> >
> > > ajuanpi+dev@
> >
> > > &gt;
> > > > wrote:
> > > > > On Tue, Oct 16, 2012 at 9:04 PM, Erik Kjellson
> > > > > &lt;
> >
> > > address@hidden
> >
> > > &gt; wrote:
> > > > > > Hello,
> > > > > >
> > > > > > I have written an implementation of the Matlab function 'importdata'.
> > > > I'm
> > > > > > getting a bit confused about the borderline between Octave Core and
> > > > > > Octave-Forge, so I don't know where it belongs...
> > > > > >
> > > > > > My implementation covers most of the functionality in the Matlab
> > > > version,
> > > > > > but not all. For example, I haven't implemented the automatic
> > > > detection
> > > > of
> > > > > > the column separator. Also, for some special cases (like if you in the
> > > > data
> > > > > > section of the file would have some letters among numeric data) you
> > > > still
> > > > > > get a slightly different output.
> > > > > >
> > > > > > The 'importdata' is, as far as I know, a part of Matlab Core rather
> > > > than a
> > > > > > toolbox.
> > > > > > Does the implementation have to be complete to be able to have it in
> > > > Octave
> > > > > > Core?
> > > > > >
> > > > > > Or is it better to add it to the Octave-Forge project for now?
> > > > > >
> > > > > > regards,
> > > > > > Erik
> > > > >
> > > > > Hi Erik,
> > > > >
> > > > > Cool!
> > > > >
> > > > > I would say that if compatibility is not complete then better to keep
> > > > > it in Forge.
> > > >
> > > > Nah, we have tons of incomplete compatibility functions in core. One
> > > > more will do no harm. It should go in core.
> > > >
> > > > > also, you should upload your function seomwhere so we can
> > > > > see the coding style, Octave core is very strict in this.
> > > >
> > > > This is a minor thing. I can adapt its style as necessary.
> > > >
> > > > - Jordi G. H.
> > > Hello,
> > >
> > > Now I have uploaded my implementation here:
> > > https://sourceforge.net/p/octave/feature-requests/41/
> > >
> > > I've noticed that a lot of function files have some kind of tests in the
> > > end of the file. I guess that you use them to check that you haven't
> > > introduced any logic error in the code when modifying it?
> > > I did find this page: http://wiki.octave.org/Tests
> > > But I guess I need to fake input from a fake file in some way, is there
> > > anywhere I can read more on how to do that?
> >
> > While perusing importdata.m in the log, I saw that it includes an error
> > message for WK1 and other spreadsheet file formats.
> >
> > While there is no WK1 support in core Octave (nor for any other spreadsheet
> > format, for that matter) there is in the Octave-Forge io package. Using that
> > and e.g. LibreOffice one can read a plethora of file formats, not just
> > .wk1/.xls/.xlsx, but also .sxc/.ods/.fods/.uods/.dbf/.html/.csv/.pxl/....
> > etc.
> >
> > Just like in e.g. usejava.m a while ago, we stumble upon core Octave
> > functions yielding messages that ignore possibly present functionality in
> > installed (and loaded) octave-forge packages.
> >
> > So, would it be a good idea to amend importdata.m to first check for
> > additional file format reading capabilities outside core Octave and only
> > when installed Octave-Forge packages don't have it, issue a message like:
> >
> >    "importdata: install and load io package and dependencies for file format
> > "<EXTENSION>
> > ?
> >
> > FYI, in this case the check for additional file format capability is easily
> > accomplished using a try...catch around a call to chk_spreadsheet_support.m
> > (from the io package); that function's output indicates what file formats
> > are supported in the io pkg.
> >
> > So what's the opinion of other devs here as to:
> >
> > 1. error messages about lacking functions in core that might be present in
> > installed octave-forge packages?
> >
> > 2. (before I start coding) amending importdata's capabilities?
<snip>
>
> If you ask me, I think that including importdata in core so directly
> wasn't really good. As I see it one of the roles of Octave Forge is to
> provide a time for testing and eventually merge it into core.

Testing etc. is what dev branches and release candidates are meant for 
as well.

> Is it possible to add the mentioned functionality in the io package in
> core?

Maybe csv2cell, cell2csv and its siblings. Those are very useful 
functions because they allow for mixed type (numeric and text) I/O 
from/to csv.
IMO the spreadsheet stuff had better wait until Java support is 
integrated in core (some time, some day).

> In that way we re-activate the OctaveForge -->  core flow (and we
> make a better Octave!)

What do other devs think?

> Also, if there is a desired to add more functions to core the first
> place to look is to globaly used, well tested packages in Forge! :D

Oh I agree. Core and OF could have a tighter relationship.

But (given the average number of active maintainers over a longer 
period) there's also something to say for keeping core Octave compact 
and well-maintained, and leaving specialized functions in OF.

Philip