[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
auto-connect fails to parallelize independent processes
From: |
Olivier Dion |
Subject: |
auto-connect fails to parallelize independent processes |
Date: |
Thu, 09 Jun 2022 20:56:27 -0400 |
Hi,
Given the following example:
--8<---------------cut here---------------start------------->8---
process proc (with output)
packages "coreutils"
outputs : file output
# {
for i in $(seq 10); do
date >> {{outputs}}
sleep 1
done
}
workflow parallel
processes
auto-connect
proc "A.out"
proc "B.out"
--8<---------------cut here---------------end--------------->8---
We can see that both processes are completely independent, and yet they
are batched linearly.
If we re-write the workflow with `graph' instead:
--8<---------------cut here---------------start------------->8---
define A : proc "A.out"
define B : proc "B.out"
workflow parallel
processes
graph
A ->
B ->
--8<---------------cut here---------------end--------------->8---
we do get parallel execution of the processes.
--
Olivier Dion
oldiob.dev
- auto-connect fails to parallelize independent processes,
Olivier Dion <=