Re: Guix on macOS

[Chris Marusich]

Hi Guix,

I've looked further into the possibility of running Guix on macOS.  I'm
convinced now that running Guix natively on macOS (i.e., compiling it
for x86_64-apple-darwin) isn't a feasible goal at this time.  GNU libc
doesn't target Darwin, so it can't be used as-is to compile programs to
run on macOS, and as Ricardo mentioned, it seems there are currently no
viable ports of GNU libc that target Darwin.  In addition, I checked the
NixPkgs source code and spoke with a Nix developer who helped get Nix
working on macOS, and I can confirm that the following are true:

* In NixPkgs, some impurities are explicitly allowed here and there to
  make Darwin builds succeed.  Specifically, certain macOS system files
  are exposed to and used by the build processes.  This means that not
  all Darwin package builds are pure, and I'm not sure what the
  licensing implications might be.

* Nix's Darwin bootstrap tools are maintained in Nix, but they aren't
  cross-compiled (e.g., from x86_64-linux).  They're built from a
  previous generation of the stdenv for Darwin.  I suspect, but do not
  know for sure, that originally the bootstrap binaries were compiled
  using the usual impure and non-free macOS toolchain.

* Nix doesn't use GNU libc for Darwin software.  Instead, it uses
  libSystem's libc.  In addition, I'm told that Nix relies pretty
  heavily on opensource.apple.com, and I'm told that many of those
  pieces of software don't have clear free software analogs.

For these reasons, it seems infeasible at this time to port Guix to
macOS natively while satisfying the two criteria that Ludo mentioned,
which were: "(1) this can be achieved in a maintainable way, and (2) the
targeted user-land software is free and buildable from source."  That's
unfortunate, but at least I did my homework, and now we know.

In lieu of running Guix natively on macOS, I'm now interested in adding
a "guix system docker-image" feature which would enable us to invoke a
command like "guix system docker-image my-os-config.scm" and receive as
output a docker image containing the GuixSD system defined by
my-os-config.scm.  This seems potentially very useful.  If we had this
feature, then anyone capable of running Docker images could run GuixSD.
This is similar to making a VM image or a regular disk image.  Docker is
popular, so I think it makes sense to make it easy for people to
experiment with GuixSD using Docker, too.

Personally, I'd like to empower people to use GuixSD on macOS in a
docker container to manage installed software in a way that is,
hopefully, almost as good as running Guix natively.  However, I can
imagine other uses for this feature, as well.  If you're a DevOps shop
and you're already heavily invested in the Docker ecosystem, this could
make it possible to deploy GuixSD easily in your environment.  For the
use case of managing software on a laptop, I think it'd be OK to mutate
the state of a GuixSD docker container by installing packages in it.
Hopefully the container can be integrated well enough to make it feel
like you've actually installed Guix on your laptop.  For a production
service, though, I would not want to deploy stateful Docker containers;
I'd prefer to use stateless containers if possible.

If we had the ability to build a docker image from a GuixSD operating
system configuration file, then it could potentially accommodate both
needs: a person could use a stateful GuixSD docker image on their laptop
for day to day tasks, and they could also leverage their company's
existing Docker infrastructure to deploy (hopefully stateless) GuixSD
docker images to production.  Instead of writing a Dockerfile based on a
base image blob that might be difficult to audit, validate, or
reproduce, and instead of compounding the issue by adding "layers" on
top of your base image blob like Docker encourages its users to do,
you'd write a GuixSD operating system configuration file to
declaratively define the service(s) in your GuixSD docker image.  You
would then build precisely the image you want, every time, using Guix.
You would then deploy it using your existing Docker tools.  This is
nothing new for GuixSD users, really, but Docker is popular and being
used today by many companies and individuals, so it might be a good way
to encourage adoption of GuixSD, Guix, the functional software
deployment model, and free software.

I've tried to implement such a feature, but I'm stuck.  In the attached
patch, I've added a command that lets you do "guix system disk-image -t
docker my-config.scm".  I've tried to merge some of the logic for
running a gexp in a linux vm with the logic for building a docker
container, but I can't get it to work.  I can successfully build Guix
with "make -j", but when I run it, I get the following error:

--8<---------------cut here---------------start------------->8---
$ ./pre-inst-env guix system disk-image -t docker gnu/system/install.scm
...
Welcome, this is GNU's early boot Guile.
Use '--repl' for an initrd REPL.

loading kernel modules...
[    1.151988] usbcore: registered new interface driver usb-storage
[    1.158506] usbcore: registered new interface driver uas
[    1.169030] hidraw: raw HID events driver (C) Jiri Kosina
[    1.174144] usbcore: registered new interface driver usbhid
[    1.175537] usbhid: USB HID core driver
[    1.221664] isci: Intel(R) C600 SAS Controller Driver - version 1.2.0
[    1.253086] ACPI: PCI Interrupt Link [LNKC] enabled at IRQ 11
[    1.282881] ACPI: PCI Interrupt Link [LNKD] enabled at IRQ 10
[    1.312644] ACPI: PCI Interrupt Link [LNKA] enabled at IRQ 10
[    1.345843] FS-Cache: Loaded
[    1.362140] 9pnet: Installing 9P2000 support
[    1.366118] 9p: Installing v9fs 9p2000 file system support
[    1.368730] FS-Cache: Netfs '9p' registered for caching
configuring QEMU networking...
loading '/gnu/store/jy509dgcsz82y13fmizv2sqaj90s1vfg-linux-vm-loader'...
ERROR: In procedure dynamic-link:
ERROR: In procedure dynamic-link: file: 
"/gnu/store/hwygv5jwd47amhp1m67iy3bkvxqjlbhm-libgcrypt-1.8.1/lib/libgcrypt", 
message: "file not found"

Entering a new prompt.  Type `,bt' for a backtrace or `,q' to continue.
GNU Guile 2.2.2
...
--8<---------------cut here---------------end--------------->8---

What's wrong?  How can I fix it?  I'm stuck and need help.

This appears to be an error on the build side, inside of the linux VM.
It seems /gnu/store/...-libgcrypt-1.8.1/lib/libgcrypt.so is not
available to the code running inside the VM.  The shared object file
(with the .so extension) exists in my store on the host side.  As far as
I can tell, I'm correctly adding libgcrypt to the gexp's closure of
inputs, so I don't understand why the error is happening.  Any tips on
how to debug this or fix it would be great.  Part of the reason I'm
stuck might be because I don't really understand how (guix config) is
used or why the (guix docker) module needs to override it.  I just
cargo-culted that part of the code in an attempt to quickly get
something that worked.  I plan to clean up the patch a lot later.

The attached patch should apply cleanly to commit
3b2fa4787938a408fab27ef7b3bc1302b6b6a805.

Thank you,

-- 
Chris

0001-Try-to-build-a-GuixSD-docker-image-currently-fails.patch
Description: Text Data

signature.asc
Description: PGP signature