Re: HDF5

[Marcus G. Daniels]

BJ> A user mentioned that HDF5 is supported by Swarm.  Would it be okay 
BJ> to add Swarm to the "What Software uses HDF5?" page, off of
BJ> our home page?

Please do!  We are using HDF5 as a our primary object serialization
backend in Swarm.  The support, especially in the pending minor
release of Swarm, is fairly complete, and many of the features of
HDF5 are used.  For example, with shallow object serialization HDF5
Compound Types are used, and for deep serialization hierarchies of
HDF5 Groups are used.  HDF5 Attributes are used to store string
tables, map keys, and `create-time' parameters.

Swarm is a simulator that provides fine-grained, composable
scheduling, and is eventually intended work in both distributed and
multithreaded shared-memory configurations.  Eventually we expect
object storage in HDF5 will be a natural substrate for parallel, high
performance random read/write access to objects in Swarm models.

Swarm has another backend that reads and writes objects in Scheme, so
Swarm can be used to convert HDF5 to and from Scheme.  This can be
used to get more detail than `h5dump', although it is somewhat
constrained by our serialization conventions.  (For small objects,
where HDF5 compression doesn't really pay this backend is a bit more
lightweight, and is more amenable to `objects over a wire'.)

It's also worth noting that the HDF5 support in `R' was designed to
read and write objects serialized with the HDF5 Swarm archiver.
Reading Swarm objects written with shallow serialization creates R
`dataframes' and deep serialization objects are loaded as tagged
lists.  R has a convenient variable syntax for accessing these tagged
lists.  For example,

   object$component1$component10$component100 

gives you instance variable `component100', which is an instance variable
in object `component10', which is an instance variable in object
`component1', which is a slot in object `object'.


                  ==================================
   Swarm-Support is for discussion of the technical details of the day
   to day usage of Swarm.  For list administration needs (esp.
   [un]subscribing), please send a message to <address@hidden>
   with "help" in the body of the message.