Prototype for using Guix with python packages

[Christopher Baines]

For a few months now, I've been assembling a prototype for how packages
could be produced for software released as Python source distributions
(sdists) [1].

I wanted to build something that was as useful as any similar tool. The
functionality offered by pip [2] was something I looked at, as through
it you can use a large amount of software from the main PyPI [3],
including many different versions of the same project and use the
setuptools feature of extras [4] which allow for optional dependencies.

This differs from the functionality currently offered by the python
packages found in the Guix repository currently, as the number is much
smaller than available on PyPI, and typically only one release is available.

I've now managed to build a system which in a very automated way can
acquire sdists, determine there dependencies, use a full dependency
resolver to determine exactly what version of each dependency to use
across the entire build graph, and convert that graph in to Guix package
records, which can then be built.

Due to the wonderful power of Guix, this should be reasonably easy to
try out. I have imported a piece of software called sentry (as it was
something I was interested in using at the time) and all of its
dependencies in to the repository at [5]. You can find the instructions
here [5] for generating a package for sentry from this information.

My plan was to work towards something that could be run automatically to
stay up to date with a PyPI, so I tried to even automatically solve as
many issues as possible, e.g.  dependency looks, so if you look at the
build graph for sentry (attached, and the command to generate it is here
[5]), you should be able to see that the tests are run for setuptools,
something that is not even done for the package in the Guix repository.

I don't view this as a good long term approach, as I'm really hoping
that maybe one day, Guix could help to reduce the specific nature in
which software written in different languages is packaged/released. But
until that happens, I think something like this could be very useful at
getting some of the value that Guix provides quickly, without lots of
manual effort.

The code is very rough as I took a lot of shortcuts to get things
working, and I have lots of ideas for things I would like to change and
improve, but as I have been working on this for so long, I thought it
was time to publish it.

I will also try to look at how what I have been doing could be used to
improve the Guix PyPI importer and the Python build system, as well as
submit the packages that are currently just in the guix-env.scm file in
the repository (pyguile, libsolv) for inclusion in Guix.

I'm quite excited by the possibilities offered by approaches like this,
and was wondering if anyone has feedback, questions, opinions, or if
anyone is working on something similar?

Thanks,

Chris

1: https://docs.python.org/2/distutils/sourcedist.html
2: https://pypi.python.org/pypi/pip
3: https://pypi.python.org/pypi
4:
http://setuptools.readthedocs.io/en/latest/setuptools.html#declaring-extras-optional-features-with-their-own-dependencies
5: http://git.cbaines.net/guix-python-integration-project/about/

sentry-graph.pdf
Description: Adobe PDF document

signature.asc
Description: OpenPGP digital signature