parallel
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Poor job distribution?


From: Fratkin, Allyn
Subject: Poor job distribution?
Date: Mon, 6 May 2013 18:16:43 +0000

First let me say thanks to Ole Tange for writing such a useful program.

I do have one issue that makes me wonder if I am doing something wrong or if it 
is a bug in parallel.

If the number of jobs I run is less than the number of job slots (hosts * jobs 
per host) then parallel seems to do poorly at distributing the jobs.  I would 
hope for an even distribution but I don't get that at all.

An example:
7 hosts, 3 jobs per host = 21 job slots
11 jobs

I would have hoped that each host got at least 1 job, with 4 getting two.  But 
here's what happened:
host 1: 3 jobs
host 2: 0 jobs
host 3: 2 jobs
host 4: 0 jobs
host 5: 3 jobs
host 6: 3 jobs
host 7: 0 jobs

If it seems crazy to have 21 job slots and only 11 jobs, it's because I use a 
fixed config of hosts and jobs slots and then feed it different sets (varying 
numbers) of jobs.

Am I doing something wrong?

I'm using GNU parallel 20130422.  The job distribution that parallel picks is 
100% repeatable.

Reproduce with:
for x in `seq 1 11`; do echo sleep 30; done | ./parallel -j 3 
-Shost1,host2,host3,host4,host5,host6,host7 --progress

Thank you. 

-- 
Allyn Fratkin                    allyn@hp.com
Hewlett-Packard Company          http://www.fratkin.com/





reply via email to

[Prev in Thread] Current Thread [Next in Thread]