Thanks for your replies. On our cluster I was successful with the
following options using slurm:
Send Espressomd-users mailing list submissions to
address@hidden
To subscribe or unsubscribe via the World Wide Web, visit
https://lists.nongnu.org/mailman/listinfo/espressomd-users
or, via email, send a message with subject or body 'help' to
address@hidden
You can reach the person managing the list at
address@hidden
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Espressomd-users digest..."
Today's Topics:
1. Re: Multiple instances of espresso on one node (Rudolf Weeber)
2. Re: Multiple instances of espresso on one node (Florian Weik)
----------------------------------------------------------------------
Message: 1
Date: Mon, 11 Mar 2019 10:41:51 +0100
From: Rudolf Weeber <address@hidden>
To: Clemens Jochum <address@hidden>
Cc: address@hidden
Subject: Re: [ESPResSo-users] Multiple instances of espresso on one
node
Message-ID: <address@hidden>
Content-Type: text/plain; charset=iso-8859-1
Hi Clemens,
On Thu, Mar 07, 2019 at 09:43:32PM +0100, Clemens Jochum wrote:
I want to optimize my use of HPC-nodes. Is it possible to have two (or
more) instances of ESPResSo running on a single node (with slurm in my
case)?
This is not Espresso specific and highly dependent on the cluster installation.
On our institute's cluster, I used the following for parallel tempering simulations, in which several Espresso instances using one core each are running on the same node and exchange information via network sockets.
The same will also work for fully independent instances of Espresso, although not all cluster operators are happy about that use.
Below script is the run script which is submitted (in this particular case on 16 nodes).
It launches the individual Espresso instances in the background (via Bash's &) and waits for completion of all background tasks at the end.
The launch command for the individual instances is probably cluster specific.
#!/bin/bash
# These may or may not be needed in a particular case
ulimit -n 4096
ulimit -u 4096
# here the Espresso instances are launched
for i in {1..256}
do
# on this particular cluster, it was necessary to disable binding of
# instances to particular cores. That may or may not be needed on other clustres.
srun -n1 --cpu_bind=no,v taskset 0xffffffff ./pypresso script.py arguments >8output-rank-$i.out &
sleep 0.1
done
wait
On a different system, it looked like this:
for i in {1..16}
do
mpirun -np 1 --bind-to none pypresso script.py arguments >$i.out 2>$i.err &
done
wait
Some clusters offer queues for smaller jobs with shared usage of a node. If that is available, it should be used for individual jobs.
Also note that Espresso's performance tends to be limited by memory bandwidth rather than by computing power, so you might not get a linear performance gain when running more instances.
Hope that helps.
Regards, Rudolf
--
Dr. Rudolf Weeber
Institute for Computational Physics
Universit?t Stuttgart
Allmandring 3
70569 Stuttgart
Germany
Phone: +49(0)711/685-67717
Email: address@hidden
http://www.icp.uni-stuttgart.de/~icp/Rudolf_Weeber
------------------------------
Message: 2
Date: Mon, 11 Mar 2019 10:53:46 +0100
From: Florian Weik <address@hidden>
To: address@hidden
Subject: Re: [ESPResSo-users] Multiple instances of espresso on one
node
Message-ID:
<address@hidden>
Content-Type: text/plain; charset="utf-8"; Format="flowed"
Hi,
typically the MPI runtime controlls the application via environment
variables,
which can lead to interference between different processes if they are
starten
within the same environment, as in your case. Maybe your problem can simply
be fixed by running srun in a sub-shell? by adding parentheses, like so
?? (/srun -n 1 --exclusive --mem=1G /path/to/pypresso /path/to/script)&//
/
?? (/srun -n 1 --exclusive --mem=1G /path/to/pypresso /path/to/script)&//
/
/
/
/Hope that helps,/
/Florian/
/
/
On 3/7/19 9:43 PM, Clemens Jochum wrote:
Dear ESPResSo-Team,
I want to optimize my use of HPC-nodes. Is it possible to have two (or
more) instances of ESPResSo running on a single node (with slurm in my
case)?
I tried the following lines:
/srun -n 1 --exclusive --mem=1G /path/to/pypresso /path/to/script &//
//srun -n 1 --exclusive --mem=1G
/path/to/pypresso//////path/to/script /&//
//wait/
Butthe programs are clearly running in serial and I'm getting the
following warning inbetween:
/Attempting to use an MPI routine after finalizing MPI//
//srun: error: n22-032: task 0: Exited with exit code 1//
//srun: Job step created//
/
Best,
Clemens
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.nongnu.org/archive/html/espressomd-users/attachments/20190311/60cf04fc/attachment.html>
------------------------------
Subject: Digest Footer
_______________________________________________
Espressomd-users mailing list
address@hidden
https://lists.nongnu.org/mailman/listinfo/espressomd-users
------------------------------
End of Espressomd-users Digest, Vol 98, Issue 6
***********************************************