[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Parallel File I/O
From: |
Arno Candel |
Subject: |
Parallel File I/O |
Date: |
Wed, 28 Aug 2002 14:14:32 -0100 |
User-agent: |
Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.0.0) Gecko/20020529 |
Hi,
Is there a clever way to handle large distributed Array I/O to disk? I
don't want all contexts to block each other while reading/writing.
A straight-forward reader implementation like
Array<3, double, MultiPatch<GridTag,Remote<Brick> > A;
A.initialize(Domain, Partition, DistributedTag());
for i=A.domain()[0].first() to A.domain()[0].last()
for j=A.domain()[1].first() to A.domain()[1].last()
for k=A.domain()[2].first() to A.domain()[2].last()
{
my_ifstream >> value;
A(i,j,k) = value;
}
scales miserably when reading a small 18 MB input ASCII file due to
blocking and communication:
1 node: 15.5176 s
2 nodes: 84.5347 s
4 nodes: 212.838 s
6 nodes: 798.704 s
12 nodes: 1538.78 s
I would appreciate any suggestions for using exisiting (binary) parallel
I/O tools provided by Pooma r2.4.0
"IO/FileSetReader.h" and "IO/FileSetWriter.h" seem to have problems with
the latest release.
Thanks,
Arno Candel
- Parallel File I/O,
Arno Candel <=