The future of 'guix environment'

[Thompson, David]

Hi all, been awhile!

I wrote the initial version of 'guix environment' almost 2 years ago,
and have used it constantly since.  It was a great start, but from my
own experiences and feedback from others, I have identified a number
of areas that need improvement.  Fixing them will require breaking
changes, but I think it's important to break things sooner rather than
later because after 1.0 it will be a hard sell.

By far the most annoying thing is that leaving and coming back to a
project (say, after a reboot) may entail rebuilding your entire
environment if you have updated Guix in between.  What was supposed to
be a quick hack turns into a long Guix maintenance session,
potentially building things from source and dealing with build
failures.

As an avid Emacs user, I want to do everything in Emacs, but if I want
to use, say, M-x compile but in the context of my development
environment, I have to wrap the command in 'guix environment' and now
everything takes way longer because Guix wastes tons of time talking
to the daemon to build a profile that I know already exists.   This
goes hand in hand with the previous issue.  We need a cache.

Another issue is that 'guix environment' doesn't have any sane
defaults to speak of.  In the Ruby world, the 'guix environment'
equivalent is called Bundler.  When you want to work on someone's Ruby
project, you enter the source tree and run 'bundle'. No other
arguments.  It fetches everything you need based upon a configuration
file, which by convention is named 'Gemfile'.  There's no good reason
why 'guix environment' cannot be this simple to use.

The default behavior of 'guix environment foo' creating an environment
with the *dependencies* of the package 'foo' seemed like a good idea
at the time (it's what nix-shell did, after all), but most of the time
when I define throwaway environments on the command-line (as opposed
to a guix.scm file) I *always* use the --ad-hoc flag.  Using the
packages themselves rather than the dependencies should be the
default.

The final issue is more of a wishlist thing, but I think it is
something that will make 'guix environment' far and away the most
powerful development environment setup tool there is.  We should be
able to define more than just what packages to include in the
environment.  We should be able to define services, too.  A great
use-case for such a feature is a web application.  Think about how
convenient it would be if a newcomer to a project could run 'guix
environment' and have nginx and mysql automatically launched for them
with the proper configuration.  The closest tool to this today would
be Docker's 'compose' tool, which can manage many containers, each
running a single service.  Surely we can do better than that!

If you've followed along this far, great!  Now here's my list of
proposed changes:

1) Add a caching mechanism.  The environment profile should get built
once, and then a symlink to it should be created in $PWD and
registered as a GC root.  This will, of course, require re-using some
'guix package' code to delete the profile.  For the sake of the rest
of this post, let's say that a --cache flag does this, and a --update
flag forces a rebuilding of the cached profile.

2) Make "ad-hoc" the default.  Remove the --ad-hoc flag and replace it
with a --dependencies flag instead, reversing the current behavior.
'guix environment --dependencies guix' would create a guix dev
environment, for example.

3) Change how --load behaves.  Instead of evaluating a file and
expected a package object in return, instead expect a
<guix-environment> record.  This would provide a declarative way to
specify the complete environment: which packages to pull in, whether
to purify the environment (--pure), whether to run in a container
(--container), whether to give the container network access
(--network), etc.  Command line flags would take precedence over what
is specified in the config file.  --load may only appear *once* in the
command line args, whereas now it many appear N times.

4) Make 'guix environment' with no other args operate like 'guix
environment --cache --load=guix.scm'.  'guix.scm' is a placeholder
name for whatever we decide the conventional name for an environment
config should be.  Throwaway environments (such as 'guix environment
ruby -- irb') would not have caching turned on by default, because
that would quickly become a burden.

5) Add support for Shepherd services.  When an environment is created,
a new Shepherd process is launched to handle all the necessary
services.  For a web application this could be nginx, mysql, redis,
etc.  This feature can be implemented later, even post 1.0, provided
we agree upon using a <guix-environment> record type.

After all these changes, a Guix user should be able to jump into a new
project, run 'guix environment' and be ready to go.  When they update
Guix and want to refresh their environment, they would run 'guix
environment --update'.

Thoughts?

- Dave