using parallel for remote host processing

[Maracic, Jay]

Hi

 

When using parallel to execute a process on a set of remote hosts, the process is prompted  by ssh ;  as this is run as a background task w no attached terminal, there is no execution of the desired process on any remote host:

 

parallel: Warning: $SHELL not set. Using /bin/sh

Could not create directory '/ms/dist/optimus/PROJ/prodid/optdsqa/common/.ssh'.

Host key not found from the list of known hosts.

You have no controlling tty.  Cannot read confirmation.I get  prompted

 

 

 

So what I want is to be able to pass the following ssh options into  parallel:

 

-o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null

 

 

Parallel complains when seeing these options:

 

###     my $rc = `head -n2  $hostsfile | /ms/dist/fsf/PROJ/parallel/20120522/parallel -v --nonall -j 1 --sshlogin -  -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null    "/ms/dist/ets/PROJ/clean-colo/qa/bin/clean.pl --group optimus"`;

 

 

I have modified parallel to include the options, and this works as desired.

 

sub sshcommand_of_sshlogin {

    # 'server' -> ('ssh -S /tmp/parallel-ssh-RANDOM/host-','server')

    # 'user@server' -> ('ssh','user@server')

    # 'myssh user@server' -> ('myssh','user@server')

    # 'myssh -l user server' -> ('myssh -l user','server')

    # '/usr/bin/myssh -l user server' -> ('/usr/bin/myssh -l user','server')

    # Returns:

    #   sshcommand - defaults to 'ssh'

    #   login@host

    my $self = shift;

    my ($sshcmd, $serverlogin);

    if($self->{'string'} =~ /(.+) (\S+)$/) {

        # Own ssh command

        $sshcmd = $1; $serverlogin = $2;

    } else {

        # Normal ssh

        if($::opt_controlmaster) {

            # Use control_path to make ssh faster

            my $control_path = $self->control_path_dir()."/ssh-%r@%h:%p";

            $sshcmd = "ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -S ".$control_path;

            $serverlogin = $self->{'string'};

            my $master = "ssh -MTS $control_path $serverlogin sleep 1";

            if(not $self->{'control_path'}{$control_path}++) {

                # Master is not running for this control_path

                # Start it

                my $pid = fork();

                if($pid) {

                    $Global::sshmaster{$pid}++;

                } else {

                    ::debug($master,"\n");

                    `$master`;

                    ::wait_and_exit(0);

                }

            }

        } else {

            $sshcmd = "ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null "; $serverlogin = $self->{'string'};

        }

    }

    $self->{'sshcommand'} = $sshcmd;

    $self->{'serverlogin'} = $serverlogin;

}

 

 

I am wondering if parallel can accept these ssh options thus obviating the need for modifying parallel?

 

Thanks for any assistance…..

 

 

 

Jay Maracic

 

---

NOTICE: Morgan Stanley is not acting as a municipal advisor and the opinions or views contained herein are not intended to be, and do not constitute, advice within the meaning of Section 975 of the Dodd-Frank Wall Street Reform and Consumer Protection Act. If you have received this communication in error, please destroy all electronic and paper copies and notify the sender immediately. Mistransmission is not intended to waive confidentiality or privilege. Morgan Stanley reserves the right, to the extent permitted under applicable law, to monitor electronic communications. This message is subject to terms available at the following link: http://www.morganstanley.com/disclaimers If you cannot access these links, please notify us by reply message and we will send the contents to you. By messaging with Morgan Stanley you consent to the foregoing.