Re: Running >10 jobs in parallel on the same remote machine

[Ole Tange]

On Fri, Aug 3, 2012 at 9:25 PM, Thomas Sattler
<sattler@med.uni-frankfurt.de> wrote:
> Am 03.08.2012 17:52, schrieb Ole Tange:
>>> How about the "Control" feature of (open-)ssh?
>
>> Can you test if ControlPersist works for this:
>>
>>     seq 100 | parallel -j0 ssh server 'echo {};sleep 2'
>
> I have almost no problems. ('parallel' is to fast, and
> the control channel needs some time to be initialized)
> So I get several
>
> "ControlSocket already exists, disabling multiplexing"

Sounds promising.

> But in case the control channel has been established
> in advance, there are no error messages. So I think
> the following should work:
>
> let parallel
>  - run one "extra" ssh to the target host
>  - give it time to establish its connections
>  - run all regular ssh's through this
>  - close the "extra" ssh when done
>
> This should work on all systems that have the "basic"
> ControlMaster feature.

I would like to have people test the idea before I implement it. I
need your help with the options:

  ssh <<what options here?>> server sleep 10000000 &
  sleep 3 # To make sure the above is logged in and set up
  seq 1000 | parallel -j0 ssh <<what options here?>> server 'echo {};sleep 2'
  kill %1 # Kill the sleeping ssh

When we have the options, I will encourage all users to try it out and
report back any problems - I am particularly interested in non-linux
machines, old distributions, slow machines and slow connections, as
that is where I anticipate problems.


/Ole