Hi there,
I use Cygwin (updated to latest packages) with the latest parallel version (20160622).
My workflow looks like this:
cat input.txt | parallel --pipe -N64 --blocksize 63K --joblog joblog.txt --retries 3 --progress python myscript.py
myscript.py does some CPU-bound processing with some network I/O and takes about 3 seconds per input line. input.txt has about 360k lines.
The above command works well for about 30-60 minutes, fully utilizing 16 cores. But then, it stops with
"Signal SIGCHLD received, but no signal handler set."
on STDERR. I tried to simulate the command with
seq 360000 | parallel --pipe -N64 --blocksize 63K --joblog joblog.txt --retries 3 --progress sleep 3
But I could not replicate the error yet.
Does anyone have an idea how to debug/resolve this?
Thanks
Stephan