parallel stops working for no obvious reason

[Thomas Sattler]

Hi there ...

I run parallel to distribute ImageMagick on a 32 Core machine.
At first 'convert' failed and google suggested not to use
debian's pre-compiled binary:

 libgomp: Thread creation failed: Resource temporarily unavailable
 open3: fork failed: Resource temporarily unavailable at
/usr/bin/parallel line 3587

After compiling ImageMagick from source (without libgomp), it
seemed OK, but parallel eventually stopped starting new jobs.

This happend last week with v20120222, I'll give it another
try with v20120322 today.

In case you want to reproduce it, here is how:

 $ time parallel -tj256 mk_gradient {1}{2}{3}{4}{5}{6} ::: \
 > {0..9} {A..F} ::: {0..9} {A..F} ::: {0..9} {A..F} ::: \
 > {0..9} {A..F} ::: {0..9} {A..F} ::: {0..9} {A..F} 2>&1 | nl

with mk_gradient:

 #!/bin/bash

 [ -s "$1.jpg" ] && exit 0
 exec convert -size 1600x1200 gradient:#$1 -gravity Center \
   -pointsize 100 -draw "text 0,0 '#$1'" $1.jpg

While PARALLEL was set to

  --load 100% --nice 10 --noswap --workdir /scratch

and the system was completely idle.

Please keep in mind that parallel will run several hours before
hitting the issue and that several GB of files will be created.

Thomas