Re: [h5md-user] Variable-size particle groups

[Peter Colberg]

Hi Olaf,

Welcome to the H5MD list!

On Tue, May 29, 2012 at 11:19:44AM +0200, Olaf Lenz wrote:
> This is my first contribution here, therefore let me quickly introduce
> myself: I'm Olaf Lenz, maintainer of the MD package ESPResSo
> (http://espressomd.org), intended for coarse-grained MD simulations. I'm
> working at the Institute for Computational Physics at Stuttgart
> University, Germany. I learned about H5MD from Felix Höfling, with whom
> I am currently giving a course on soft matter research.
> 
> H5MD is compelling to us, as we are working on coarse-grained
> simulations, and compared to atomistic simulations, we need a lot of
> flexibility in a format to store our simulations, as basically
> everything can change throughout a coarse-grained simulation.
> 
> After Peter's mailing, I have had a first thorough look at H5MD. From
> our point of view, Peter describes a very valid point - the number of
> particles in a simulation can vary. This is not only true in the case of
> grand-canonical simulations, but also other state-of-the-art-schemes
> have a varying particle number, e.g. the ADResS-scheme, where the level
> of detail might vary in different regions. As an example, think of a
> protein-water simulation, where the protein and the surrounding nm of
> water is simulated on an atomistic level of detail, while the water
> further away is simulated on a coarse-grained level with less
> interaction sites per water molacule. I believe that this kind of
> schemes will become more important in the future, so allowing to store
> trajectories with varying particle number may become important.

You are spot on with coarse graining. This is exactly what I intend to do.


> On 05/29/2012 10:15 AM, Felix Höfling wrote:
> >> H5MD implements an optional dataset “range” inside each
> >> trajectory subgroup, next to the other datasets groups “step” and
> >> “time”.
> 
> Besides making the format more complex, as Felix remarked, I believe
> that forcing Peter's definition upon the format would also have major
> impact upon parallel IO.
> 
> I think a relatively simple solution to avoid making the format more
> complex while still allowing for varying particle number would be to
> specify that if the subgroup "range" exists in a time-dependent dataset,
> the subgroup "value" is to be interpreted in the way Peter described,
> otherwise it uses the simple definition.

Ok, then such a “range” dataset should be optional.

The point of parallel I/O is very interesting: How would this be
implemented in practice? To warn you, I have not used parallel
HDF5 yet.

I would assume that e.g. for a parallel MPI simulation, one would need
a designated process to extend the “value”, “step”, “time” datasets
on each time-step, after which all processes perform a write to the
their slice of the newly appended region of the “value” dataset.

Then adding a “range” dataset should not change this requirement.
There would still be a single process to extend the datasets. The
designated process would further communicate to the other processes
the new range with regard to “value”, after which all processes
perform a write to their sub-range.

Are my assumptions on parallel HDF5 I/O anything close to reality?

Regards,
Peter