Re: Alternate termination sequence option --term-seq

[Rasmus Villemoes]

On Tue, May 26 2015, Ole Tange <ole@tange.dk> wrote:

> On Thu, Apr 30, 2015 at 12:47 PM, Rasmus Villemoes
> <rv@rasmusvillemoes.dk> wrote:
>> On Wed, Apr 29 2015, Ole Tange <ole@tange.dk> wrote:
>>
>>> On Wed, Apr 29, 2015 at 2:07 PM, Rasmus Villemoes <rv@rasmusvillemoes.dk> 
>>> wrote:
>>>> On Wed, Apr 29 2015, Ole Tange <ole@tange.dk> wrote:
>>>>
>>>>> This still has the risk of killing an innocent PID and its children.
>>>>
>>>> Killing (in the sense of sending any signal whatsoever) an
>>>> innocent/unrelated PID is completely unacceptable, IMO. On a reasonably
>>>> busy system, PID reuse within 10 seconds is far from unlikely.
>
> I have tried making each process its own process group. It fails for
> two reasons:
>
>   * open3 does not support it

Yeah, it would need to support a hook function to run as the child
process, between fork() and exec(). Reading the manual of IPC::Open3
however points to IPC::Run, "This is a CPAN module that has better error
handling and more facilities than Open3.", and it indeed has

   # Calling \&set_up_child in the child before it executes the
   # command (only works on systems with true fork() & exec())
   # exceptions thrown in set_up_child() will be propagated back
   # to the parent and thrown from run().
      run \@cat, \$in, \$out,
         init => \&set_up_child;

I have no idea how easy or hard it is to use IPC::Run as a replacement
for IPC::Open3.

>   * giving a process its own process group also gives it a new
> terminal, and thus Ctrl-C does not kill the children and it breaks
> --tty. That means Ctrl-C will have to be propagated by Parallel.

Well, first of all, I think it would be reasonable to start by handling
the common case first, so one could disable all process group logic in
the --tty case. Also, the --tty case is not where I'd expect rogue trees
of subprocesses to be spawned.

> The git code now does:
>
[snip]
>
> So new pids will never be added: Only the original family will be
> killed. In practice that works really well.
>
> But it is not bullet proof.
>
[snip]
>
> After some minutes a process is wrongly killed.
>
> As far as I can tell it is because the family_pids are dynamic, and we
> cannot tell for sure if the a pid has been reused: it can happen
> milliseconds after family_pids have been computed. In a normal system
> chances are extremely slim, but as my test system shows it is
> non-zero.

Again: GNU Parallel has no business sending signals to arbitrary process
ids. As your toy example shows, this may kill an unrelated process.

> All in all this sucks.

Completely agree :-)

> I believe Rasmus' idea of process groups is the only safe way forward,
> but it is harder for me to see, how that can be implemented: Wrapping
> it in a perl oneliner will require GNU Parallel to forward signals
> (which signals?) to the process groups. And if a --tty is requested,
> this will have to be disabled.

Proper handling of --tty would probably involve some playing with
tcgetpgrp and tcsetpgrp, so that the controlling terminal is explicitly
given to and taken from the child process (group). That way, Ctrl-C
will, while the child is running, send the signal to the child process
(group), and Parallel would then just need to react to the fact that the
child terminated due to a signal. 

Rasmus