Re: Managing environments (Python venv, guix environment, etc.)

[sbaugh]

Eli Zaretskii <address@hidden> writes:
> Then perhaps it's time to clarify what an environment really is?  I
> don't think that has been done in this discussion; if I missed
> something, please point that out.

An environment is a collection of environment variables paired with
values for those variables. A program is said to run in the environment
when it runs with those environment variables set to those values. Other
environment variables might be set to arbitrary values.

A program is running in a (making up some jargon for this thread):
- "pure environment" if the only environment variables set are those
  specified in the environment.
- "combined environment" if environment variables are set to a
  combination of values from two different environments; generally, a
  combination of a custom environment and the pre-existing
  environment. The custom environment values are generally prepended to
  the pre-existing environment values, which means the custom
  environment search paths will be searched first.

A program running in a Unix distribution uses some environment variables
("search paths") to locate distribution resources and some environment
variables ("configuration variables") to determine its
configuration. For example, PATH is a search path for executables,
LIBRARY_PATH is a search path for libraries, and LANG is a configuration
variable containing the name of the current locale.

The typical Unix distribution has some default environment in which
programs run to access the default resources and configuration of the
distribution.

The "pre-existing environment" is the one which a program started in;
generally this is inherited from the program's parent process without
modification.

An interesting "custom environment" is an environment containing
settings which are not default for the Unix distribution on which the
environment is being used. Instead, a custom environment contains
settings for search paths which point to directories that contain some
custom binaries and libaries, and settings for configuration variables
to further manipulate the behavior of programs running within the custom
environment.

Custom environments are useful for development because they allow
developing and compiling programs in a portable, reproducible,
site-local-configuration-independent way. They are useful also for
distribution by allowing finnicky software to run in an environment that
is exactly what it expects. (These use cases are similar to how chroots
and containers are used, though environments are clearly lighter-weight)

Supporting the use of custom environments in Emacs is the point of this
mailing list thread. Custom environments are (generally) what I have
been referring to when I've said "environment" in this thread.

environment.el would allow using a given "custom environment" to run a
program in Emacs in either a "pure environment", or a "combined
environment" created from the custom environment and the "pre-existing
environment" of Emacs. Ideally, environment.el would provide a nice user
interface for doing that.

>> If we embed the environment into the file name, then the environment
>> also becomes a property of a file name.
>
> But since an environment is not a property of a file name, doing that
> would be a mistake, it seems.

Sorry, I don't think I was clear.

TRAMP has the function of visiting files located on remote hosts, and it
needs to identify the files located on remote hosts. To do this, it
combines an identifier for the remote host, with the filename of the
file on the remote host, and then using that combination as the filename
within Emacs. For environment.el, there is no analogous need to identify
files located inside environments, since the files "located" within an
environment are exactly the same files with the same paths located
outside the environment. (With one key exception - the meaning of ~
depends on the value of HOME in the environment. But that's minor.)

However, TRAMP also needs to be able to support running processes on
remote hosts. And there its mechanism is rather unnatural - why
should we use a filename to identify the remote host on which we want to
run a process? But we chose to do it, and I think it was a pretty good
choice which results in pretty intuitive behavior in the rest of
Emacs. Using a filename to identify the environment to run a process in
is exactly as unnatural as this use of TRAMP, but I think it will result
in equally intuitive behavior.

By the way: In future work I would like to support editing files located
inside chroots, containers, different mount namespaces, and so on.  In
all those cases it makes sense to have a magic prefix to identify where
to look, and then the file name; and in all those cases the files could
(eventually, with work in C) be directly visited on the local machine
rather than transferred over the network. So in those cases it's very
clear to me that file-name-handler-alist should be used, and since TRAMP
basically assumes transferring files over the network/through a
subprocess, a new file-name handler should be made. But all of those
things are quite similar to environments, and it would be natural to
reuse a magic prefix created for environments, to support those
things. So that's another reason why using a magic prefix would be a
sane choice for environments.