Re: input from text files

[Marcus G. Daniels]

>>>>> "JM" == Marshall James <address@hidden> writes:

JM> I know that using InFile to read from text files is not
JM> advised, but what is the recommended alternative? 

Swarm 1.4.1 knows how to serialize objects to and from Lisp and HDF5.
The intent of this feature is to factor out data handling to other
programs such as batch-mode Emacs, Guile and R.

After all, Swarm is trying to be a featureful simulator, and to the
extent it tries to be a text processing tool like Emacs or Perl5, it
will do a inferior job, and in the process distract from the basic mission.

Here is how you can use R to load a text file, save it to HDF5, and
then recover the data as an object in Swarm.  I'm showing the HDF5
usage instead of the Lisp usage mainly because I think the Lisp
archives, e.g. ~/.swarmArchiver, will be obvious to Lisp users, and,
as a practical matter, the usability of HDF5 serialization benefits
from some new features in R that folks won't otherwise know about.
Also, the HDF5 serialization is more powerful than the Lisp
serialization, e.g., HDF5 files are randomly accessible and offer data
compression.

1. Locate a data file. 

  $ cat data.txt
  val1,val2,str
  3,4,Hello World
 
2. Start R in the home directory. 
  
  $ cd
  $ R

  For the time being, we've got a custom version of R at SFI that is
  available from:

  ftp://ftp.santafe.edu/pub/swarm/support-software

  ..but eventually we will merge with the official version, which is
  under active development, and is actively supported.
  Incidentally, the current development version has cool features like
  a GNOME graphical user interface (http://www.gnome.org).

3. Read the data file.  
   Note [header=TRUE] indicates that the first line contains field names, and
   [sep=","] indicates that the columns are separated by commas.
   [obj <-] means "assign to variable `obj'".

  > obj <- read.table("data.txt",header=TRUE,sep=",")
   
4. Set an attribute on the object to clue Swarm in to the fact you want
   the object loaded as a MyObject class.  Here we assign to an attribute
   of an object, rather than a symbol as in step #3.  

  > attr(obj,"type") <- "MyObject"

  This step can be skipped if you are prepared to access the object
  using only VarProbes -- probably more of a hassle than just creating a real
  class -- you'd get an object with a made up class name and no other methods
  besides those in CreateDrop.

5. Wrap up the object with some application metadata, and assign to
   the serialize variable.  Swarm looks for toplevel archive objects
   according to what application they belong to (the program below
   compiles to an executable called `serialize'), then by the
   mode-of-use indicator (`default'), and finally the object name.

  > serialize <- list(default=list(myObject=obj))

6. Save the archive structure (and object) created in step #5.
   Note that swarmArchiver.hdf is the default system archiver provided
   by Swarm, other instances of Archiver can be made, with other paths.

  > hdf5save("swarmArchiver.hdf",serialize)

7. Set up your Swarm model to access this data.  Below is an example program.

#import <simtools.h>
#import <defobj/Create.h>

@interface MyObject: CreateDrop
{
  // Note the variables here follow the ones in the header of data.txt.
  double val1;
  double val2;
  const char *str;
}
@end

@implementation MyObject
- print
{
  printf ("%f %f `%s'\n", val1, val2, str);
}
@end

int
main (int argc, const char **argv)
{
  id obj;

  initSwarm (argc, argv);
  
  // Get the reconstructed object named `myObject' in the `serialize/default' 
  // group of the HDF5 file.
  obj = [hdf5Archiver getObject: "myObject"];

  [obj print];
}

/*
Local Variables:
compile-command: "/opt/egcs/bin/gcc -o serialize -g -Wno-import 
-L/opt/SUNWtcl/8.0/sun4/lib -R/opt/SUNWtcl/8.0/sun4/lib -L/opt/SDGblt/2.4g/lib 
-R/opt/SDGblt/2.4g/lib -L/opt/SDGlibffi/1.20/lib -R/opt/SDGlibffi/1.20/lib 
-L/opt/SDGswarm/1.4.1/lib -L/opt/SDGzlib/1.1.3/lib -L/usr/local/X11/lib 
-R/usr/local/X11/lib -L/usr/openwin/lib -R/usr/openwin/lib 
-L/opt/SDGhdf5/1.0.1/lib -I/opt/SDGswarm/1.4.1/include serialize.m  -lsimtools 
-lsimtoolsgui -lactivity -ltkobjc -lrandom -lobjectbase  -ldefobj -lcollections 
-lmisc  -ltclobjc -ltk8.0 -ltcl8.0 -lBLT -lsocket -ldl -lnsl -L/usr/openwin/lib 
-lhdf5 -lpng -lz -lXpm -lX11 -lffi -lm -lobjc -lpthread -lposix4"
End:
*/


                  ==================================
   Swarm-Support is for discussion of the technical details of the day
   to day usage of Swarm.  For list administration needs (esp.
   [un]subscribing), please send a message to <address@hidden>
   with "help" in the body of the message.