[Fab-user] Calling fabric from within multithreaded program

[John Kelly]

Hi all,

I've written a multithreaded python script to automate the creation of a
stack of machines in AWS. Part of this script involves creating an EBS
volume and attaching it to the machine. I use boto to create and attach
the EBS volume. I then want Fabric to step in and actually do all the
heavy lifting of this EBS volume (running mkfs, editing /etc/fstab,
mounting the volume). This works fine in the non-threaded version of the
script, but in the multi-threaded version, it looks as if the threads
aren't keeping up and are stomping on each other.

Here's where we call Fabric from the script:

volume = create_ebs_volume(instance, 'sdf', default_ebs_size,
instance_name, stack_name)
time.sleep(10) # Wait 10 seconds to give the drive a chance to mount
with(settings(host_string=instance.public_dns_name,
key_filename=key_filename, user='ubuntu')):
  mount_ebs_volume('xvdf','/vol')

And the mount_ebs_volume definition:

def mount_ebs_volume(attach_point,mount_point):
  sudo("mkfs.xfs /dev/%s" % attach_point)
  sudo("echo /dev/%s %s xfs noatime 0 0 >>  /etc/fstab" % (attach_point,
mount_point))
  sudo("mkdir %s" % mount_point)
  sudo("mount %s" % mount_point)

What we're seeing is that, for the most part, the mkfs.xfs works fine on
all machines, but the mkdir and mount commands are often run on the
wrong machine. For example, if there are three threads at this part of
the script, one machine will get all three 'mkdir' and 'mount' and the
other two will get nothing.

I have a feeling this is because I'm using with
with(settings(host_string...)), but can't figure out a better way to do
this in a multi-threaded setting. Anyone got any pointers?

jk2

- - -
John Kelly - address@hidden
Tel: +353 86 6038925