Re: [Mark Meyer] Re: AWS + OpenStack support

[Chris Marusich]

Mark Meyer <address@hidden> writes:

>     Chris> Long story short, instead of starting with a base image and
>     Chris> modifying it (e.g., by injecting credentials at first boot
>     Chris> via the EC2 metadata service), one appealing alternative is
>     Chris> to use EC2's VM import feature to actually import precisely
>     Chris> the system that you want to launch:
>
>     Chris> https://aws.amazon.com/ec2/vm-import/
>
> Which does not work with GuixSD (tried it). Apparently it looks into the
> image an expects stuff like fstab. I find it not very trust building
> that it actually inspects the image.

Yes, we would need to write some code that creates the kind of image (
that EC2 expects.  The documentation claims that "raw" format is
supported; I guess they have some unspoken requirements (like the fstab
thing you mentioned) that complicate matters?

https://docs.aws.amazon.com/vm-import/latest/userguide/vmimport-image-import.html#prerequisites-image

For what it's worth, if you have a support contract (even a low tier
one) with AWS, their support engineers can help answer questions.  I'm
sure they could also help figure out what the exact restrictions on the
formats are, if the documentation is not clear enough:

https://docs.aws.amazon.com/awssupport/latest/user/getting-started.html

>     Chris> Customizations, such as SSH credentials, would be specified
>     Chris> in a GuixSD operating system configuration file and built
>     Chris> into the VM image, so neither the EC2 metadata service, nor
>     Chris> hacks like the "cloud-init" script used by some distros,
>     Chris> would enter into the picture at all.
>
>     Chris> Some preliminary work in a similar spirit was already done in
>     Chris> the branch 'wip-deploy', but I don't think it was
>     Chris> EC2-specific in any way.  Perhaps by looking there, you can
>     Chris> find some inspiration?
>
> Here the immediate downside would be that stuff like auto-scaling does
> not work out of the box. Which some people consider one of the selling
> features of AWS, the prices for VM hosting being rather high.

I think auto-scaling is orthogonal to this discussion.  If we build a
way to deploy an EC2 instance from a GuixSD operating system
configuration file, then it implies that we also have the ability to
create an AMI from the operating system configuration file.  So
auto-scaling is definitely possible.  Whether or not auto-scaling works
for a given use case depends on the use case, not on the mechanism by
which the AMI used for auto-scaling was created.

>     Chris> I think it would be better to spend your energy on creating a
>     Chris> mechanism that allows an individual to build a GuixSD image
>     Chris> from their own operating system configuration file, import
>     Chris> that into EC2, and then launch an instance from it.  If such
>     Chris> a feature were available in GuixSD, you could do it once from
>     Chris> a desktop/laptop with a slow internet connection to create a
>     Chris> "control server" in the cloud (with a fast internet
>     Chris> connection), and then you could run it from the control
>     Chris> server as needed to quickly spin up whatever other instances
>     Chris> you might need.
>
> I think the above steps could be shortened somewhat and automated, if
> you know you're running on ec2.
>
> I don't see a way to cleanly import an image into AWS. This is however
> different for OpenStack, there you have an image service that does just
> what we need.

I haven't tried it myself, but I suspect that the "VM Import" feature is
the right approach.  I think it's probably the best thing to try first.
We would need to implement the mechanism for building a GuixSD system in
whatever VM format EC2 requires, and we would need to implement a Guile
interface for EC2 (or at least wrap the AWS CLI).

> I'll try my hand at optimizing these steps on the weekend.

Good luck!  If you need anything, I'm happy to talk.

Finally, I'd like to mention that although I work at AWS, this email
correspondence is entirely my own personal opinion and does not reflect
the views or opinions of my employer.

-- 
Chris

signature.asc
Description: PGP signature