Re: [Fab-user] tricky parallel execution

[Dmitry Makovey]

Hi Jeff, thanks for reply, see reply inline below:

On September 8, 2012 16:27:37 Jeff Forcier wrote:
> > Sample as a sequence of ssh commands in two parallel sessions (desktop and
> > desktop2):
> > 
> > desktop $ ssh address@hidden "/usr/bin/postmaster -p 8888 -D /alter/data
> > -d3"
> > desktop2 $ ssh address@hidden "pgdump -U postgres -p 8888" > /tmp/dump.sql
> > desktop $ ^C
> 
> Unfortunately, right now this use case -- parallelism w/ different
> commands against a single target -- is not possible, so I'm afraid
> you're out of luck. Future versions will definitely make this easier,
> but the parallel we added to Fab 1.x was oriented towards the "do the
> same thing on multiple systems concurrently" which is distinct from
> what you're trying to do here.
> 
> You should be able to use other "backgrounding" solutions to effect
> this in Fab 1, however, e.g.:
> 
> * Spin up the postmaster in a screen session, storing its PID in a
> known location, then sending TERM or similar to that PID after the
> dump is done. (Might be possible to simply name the screen session and
> figure out if killing the screen session sends the right signal to
> postmaster -- not sure, but would be even easier than the PID
> approach.)

in case of postmaster - that is probably what I'll end up doing. However 
before commiting to that I decided to play with os.fork():

@task
def bg_check():
    """KILLME: test routine for multiprocessing"""
    child_pid=os.fork()
    if child_pid!=0:
        # parent
        run('tail -f /var/log/messages &')
        time.sleep(20)
        os.kill(child_pid,signal.SIGTERM)

    else:
        print('Oh-ho-ho')

which seems to work but something tells me this approach may bite me in the 
end if Fabric keeps track of connections by "reference counts" and does 
cleanup on SIGTERM - i.e. I'm destroying child and inadvertently terminating 
connection with the server (sorry didn't have time to go through Fabric's code 
thoroughly to get clean understanding). 

So the question is: is above code really working or should I stay away from 
doing forks within tasks?

-- 
Dmitry Makovey
Web Systems Administrator
Athabasca University
(780) 675-6245
---
Confidence is what you have before you understand the problem
    Woody Allen

When in trouble when in doubt run in circles scream and shout 
     http://www.wordwizard.com/phpbb3/viewtopic.php?f=16&t=19330


-- 
    This communication is intended for the use of the recipient to whom it
    is addressed, and may contain confidential, personal, and or privileged
    information. Please contact us immediately if you are not the intended
    recipient of this communication, and do not copy, distribute, or take
    action relying on it. Any communications received in error, or
    subsequent reply, should be deleted or destroyed.
---