01/01: website: Add post about managing servers with 'guix deploy'.

[David Thompson]

davexunit pushed a commit to branch master
in repository guix-artwork.

commit 026726b90db261088d00a489648642f865fcc28c
Author: David Thompson <address@hidden>
Date:   Mon Nov 4 08:29:23 2019 -0500

    website: Add post about managing servers with 'guix deploy'.
---
 website/posts/managing-servers-with-gnu-guix.md | 280 ++++++++++++++++++++++++
 1 file changed, 280 insertions(+)

diff --git a/website/posts/managing-servers-with-gnu-guix.md 
b/website/posts/managing-servers-with-gnu-guix.md
new file mode 100644
index 0000000..da65abd
--- /dev/null
+++ b/website/posts/managing-servers-with-gnu-guix.md
@@ -0,0 +1,280 @@
+title: Managing Servers with GNU Guix: A Tutorial
+date: 2019-11-04 22:00
+author: Jakob L. Kreuze
+tags: GSoC, Programming inferfaces, Scheme API
+---
+
+The outcome of this year's
+[GSoC](https://summerofcode.withgoogle.com/projects/#5232565294727168)
+is a Guile-based programming interface named `guix deploy` for
+automatically creating, upgrading, and changing the configurations of
+machines running the Guix System.  The tool is comparable to
+[Ansible](https://www.ansible.com/) or
+[NixOps](https://nixos.org/nixops/), but makes use of the system
+configuration facilities provided by Guix.  A [post from earlier this
+summer](http://guix.gnu.org/blog/2019/towards-guix-for-devops/)
+described an early version of the programming interface, but we're
+already a few months into autumn, so it's time for guide on how you
+can use `guix deploy` in production.
+
+#### Simple case: managing a home server
+
+If the machine you need to manage is already running the Guix System,
+it shouldn't be too hard to incorporate `guix deploy` into your
+workflow.  All that's needed is the `<operating-system>` declaration
+you've been passing to `guix system reconfigure` and some information
+about the machine (specifically its IP address and architecture). The
+`guix deploy` command is invoked with the filename of a "deployment
+specification" as an argument, whose contents should look something
+like this:
+
+```scheme
+;; Module imports
+(use-modules (gnu) (guix))
+(use-service-modules networking ssh)
+(use-package-modules bootloaders)
+
+;; Operating system description
+(define os
+  (operating-system
+    (locale "en_US.utf8")
+    (timezone "America/New_York")
+    (keyboard-layout (keyboard-layout "us" "altgr-intl"))
+    (bootloader (bootloader-configuration
+                 (bootloader grub-bootloader)
+                 (target "/dev/sda")
+                 (keyboard-layout keyboard-layout)))
+    (file-systems (cons* (file-system
+                          (mount-point "/")
+                          (device "/dev/sda1")
+                          (type "ext4"))
+                         %base-file-systems))
+    (host-name "alyssas-home-server")
+    (users (cons* (user-account
+                   (name "alyssa")
+                   (comment "Alyssa")
+                   (group "users")
+                   (home-directory "/home/alyssa")
+                   (supplementary-groups
+                    '("wheel" "netdev" "audio" "video")))
+                  %base-user-accounts))
+    (sudoers-file (plain-file "sudoers" "\
+root ALL=(ALL) ALL
+%wheel ALL=NOPASSWD: ALL\n"))
+    (services (append
+               (list (service openssh-service-type
+                              (openssh-configuration
+                               (permit-root-login #t)))
+                     (service dhcp-client-service-type))
+               %base-services))))
+
+;; List of machines to deploy
+(list (machine
+       (operating-system os)
+       (environment managed-host-environment-type)
+       (configuration (machine-ssh-configuration
+                       (host-name "alyssa-p-hacker.tld")
+                       (system "i686-linux")
+                       (identity "/home/alyssa/.ssh/server_key")))))
+```
+
+Even if Scheme isn't your forté, parts of this should look familiar if
+you've used Guix before.  The "operating system description" section
+in particular is something you might use with `guix system
+reconfigure`.  What's new is the last part: We construct a `list`
+containing one `machine` of the `managed-host-environment-type`, for
+which we've specified that `%system` is the `operating-system`
+declaration that we want to install on it, and that we can connect to
+it using the parameters specified by the `machine-ssh-configuration`.
+
+Let's take a step back for a moment and explain what a `machine` is.
+`guix deploy` aims to support a number of different use-cases, which
+we abstract as "environment types".  We'll see other environment types
+later in this article, but the general idea is that these environments
+specify how resources should be "provisioned" or created.  For
+example, an environment type designed for working with a Virtual
+Private Server (VPS) provider might make calls the provider's API to
+request a virtual machine before installing the `machine`'s
+`operating-system` declaration on it.
+
+The environment type used in this example,
+`managed-host-environment-type`, is intended for machines that are
+already running Guix System and are accessible over SSH.  It expects
+that the `configuration` field of the `machine` be an instance of
+`machine-ssh-configuration`, whose available fields are described in
+the
+[manual](https://guix.gnu.org/manual/devel/en/html_node/Invoking-guix-deploy.html).
+This gives `guix deploy` the information it needs to connect to the
+machine's SSH daemon.
+
+Running `guix deploy` with this file would build the "operating system
+closure" of `%system` -- a bundle of the packages, configuration
+files, and other dependencies necessary to realize that configuration
+-- for the architecture specified by `system` (in this case
+`i686-linux`), send it over SSH to `alyssa-p-hacker.tld`, and then
+remotely "activate" the configuration by creating a new system
+generation and upgrading running services.  Sweet!  Upgrading our
+single server setup has been reduced to an endeavour involving just
+over a dozen keystrokes.
+
+#### More advanced case: managing a virtual private server deployment
+
+One server not cutting it for you?  `guix deploy` can still help.
+Suppose we run a web service that we'd like to split up across
+multiple machines for performance reasons.
+
+```scheme
+(define %forum-server-count 4)
+
+(define (forum-server n)
+  (operating-system
+    (host-name (format #f "forum-server-~a" n))
+    ...
+    (services (append (list (service httpd-service-type
+                                     (httpd-configuration
+                                      ...)))
+                      %base-services))))
+
+(map (lambda (n)
+       (machine
+        (system (forum-server n))
+        (environment digital-ocean-environment-type)
+        (configuration (digital-ocean-configuration
+                        (region "nyc3")
+                        (size "s-1vcpu-1gb")
+                        (enable-ipv6 #t)))))
+     (iota %forum-server-count))
+```
+
+This example isn't as concrete as the first one; I'm intentionally
+omitting parts of the configuration to make the example clearer.
+Here, we automate the creation of `%forum-server-count` Digital Ocean
+"droplets" in their `NYC3` region by creating a list of 4 machines.
+
+Assuming that the environment variable `GUIX_DIGITAL_OCEAN_TOKEN` is
+properly set, running `guix deploy` with this file will do much of the
+same as the previous example.  The difference is that four virtual
+machines will be automatically created on Digital Ocean.
+
+One important thing to note about the `digital-ocean-environment-type`
+is that, currently, it *does not* automatically clean up unused
+virtual machines. If you change something in the deployment
+specification and run `guix deploy` again, the virtual machines from
+the previous deployment will remain until you destroy them yourself.
+
+#### A quick peek into the internals of `digital-ocean-environment-type`
+
+It would be an overstatement to say that the process of implementing a
+new environment type is easy, but a fair amount of the work has
+already been done for you.  We'll use the definition of
+`digital-ocean-environment-type` as an example.
+
+```scheme
+(define digital-ocean-environment-type
+  (environment-type
+   (machine-remote-eval digital-ocean-remote-eval)
+   (deploy-machine      deploy-digital-ocean)
+   (roll-back-machine   roll-back-digital-ocean)
+   (name                'digital-ocean-environment-type)
+   (description         "Provisioning of \"droplets\": virtual machines
+ provided by the Digital Ocean virtual private server (VPS) service.")))
+```
+
+The `environment-type` record specifies a small amount of metadata
+(`name` and `description`), as well as the names of three procedures:
+one for remotely evaluating a
+[G-Expression](http://guix.gnu.org/manual/en/html_node/G_002dExpressions.html#G_002dExpressions)
+on the host (`machine-remote-eval`), one for deploying an
+`operating-system` declaration to the host, and one for rolling the
+host back one generation.
+
+This might sound like a lot, but the pattern for these high-level
+environment types is to somehow obtain a machine running Guix System,
+set up an SSH daemon, and then delegate to
+`managed-host-environment-type`.  `digital-ocean-remote-eval` is a
+pretty good example of this:
+
+```scheme
+(define (digital-ocean-remote-eval target exp)
+  "Internal implementation of 'machine-remote-eval' for MACHINE instances with
+an environment type of 'digital-ocean-environment-type'."
+  (mlet* %store-monad ((name (droplet-name target))
+                       (network -> (droplet-public-ipv4-network name))
+                       (address -> (hash-ref network "ip_address"))
+                       (ssh-key -> (digital-ocean-configuration-ssh-key
+                                    (machine-configuration target)))
+                       (delegate -> (machine
+                                     (inherit target)
+                                     (environment 
managed-host-environment-type)
+                                     (configuration
+                                      (machine-ssh-configuration
+                                       (host-name address)
+                                       (identity ssh-key)
+                                       (system "x86_64-linux"))))))
+    (machine-remote-eval delegate exp)))
+```
+
+As you can see, you could reasonably go about implementing an
+environment type without ever having to learn what a G-Expression is.
+Here, `droplet-name` derives the name of the droplet from the
+machine's `operating-system` declaration, the information necessary to
+connect to the droplet is found using `droplet-public-ipv4-network`,
+and that's used to create machine of `managed-host-environment-type`.
+
+#### In conclusion
+
+I sincerely hope that `guix deploy` proves to be a useful to anyone
+dealing with system administration or software development.
+Transactional upgrades should provide peace of mind to those managing
+servers (it's worth noting that few existing tools are capable of
+recovering from failed deployments), and I believe that
+procedurally-generated deployment configurations could very well be
+the future of distribution for software such as web services: Imagine
+if setting up a [Mastodon](https://joinmastodon.org) instance were as
+easy as downloading a Scheme file and handing it off to `guix deploy`.
+The ease of writing code that generates code isn't the only benefit of
+using Guile for something like this.  Guile is a general-purpose
+programming language, so more advanced tooling can reasonably be built
+atop `guix deploy`. A GTK or Emacs DevOps interface, perhaps?  (If
+that idea sounds outlandish, consider that the latter has [already
+happened](https://emacs-guix.gitlab.io/website/) for the package
+management interface.)
+
+It's been a great summer working alongside everyone in the Guix
+community.  `guix deploy` is brand new (and a little unstable!), but
+we've had enthusiastic adoption by several on the mailing lists who
+were quick to report any issues they found.  I'd like to thank
+everyone on the `#guix` IRC channel and the mailing lists who got me
+up to speed with the code, answered my questions, gave feedback when I
+submitted my patches, and put `guix deploy` under the pressure of use
+in production.  And of course, I want to thank my mentors Christopher
+Lemmer Webber and David Thompson.  I had to make some hard design
+decisions, but this was made easier thanks to the guidance of two
+experienced Guix veterans.
+
+Oh, and this isn't a goodbye.  I really feel I've found my place as a
+Guix contributor, and I can't wait to see what the future will bring
+for `guix deploy`.  Catch ya on the mailing lists!
+
+#### Editor's note
+
+Thank you for all of your hard work, Jakob!
+
+#### About GNU Guix
+
+[GNU Guix](https://www.gnu.org/software/guix) is a transactional package
+manager and an advanced distribution of the GNU system that [respects
+user
+freedom](https://www.gnu.org/distros/free-system-distribution-guidelines.html).
+Guix can be used on top of any system running the kernel Linux, or it
+can be used as a standalone operating system distribution for i686,
+x86_64, ARMv7, and AArch64 machines.
+
+In addition to standard package management features, Guix supports
+transactional upgrades and roll-backs, unprivileged package management,
+per-user profiles, and garbage collection.  When used as a standalone
+GNU/Linux distribution, Guix offers a declarative, stateless approach to
+operating system configuration management.  Guix is highly customizable
+and hackable through [Guile](https://www.gnu.org/software/guile)
+programming interfaces and extensions to the
+[Scheme](http://schemers.org) language.