[bug#30572] [PATCH 1/7] tests: Add tests for "guix pack".

[Chris Marusich]

address@hidden (Ludovic Courtès) writes:

>> +# Use --dry-run because it takes too long to actually build everything.
>> +GUIX_BUILD_OPTIONS="--no-substitutes --dry-run"
>> +export GUIX_BUILD_OPTIONS
>> +
>> +# Build a tarball.
>> +guix pack coreutils
>
> It would be ideal if we could actually build something, but built
> something cheap.
>
> The way we do that in those tests is by:
>
>   1. Using the ‘guile-bootstrap’ package as an example, under the
>      assumption that it’s already available, does not require
>      networking, and is built in one or two seconds.
>
>   2. Using ‘--bootstrap’ or a similar option so that the derivations use
>      ‘guile-bootstrap’ instead of ‘guile-final’, for the same reason.
>
> See for instance tests/guix-package.sh.
>
> Would you be willing to try something along these lines?

That's a good idea!  I tried it (see attached patch).  Unfortunately,
even after adding a --bootstrap option, "guix pack" tries to build many
things, so tests/guix-pack.sh takes hours to run the first time you try
it (it ran for 2 hours on my laptop and then failed because gcc failed
to build for unrelated reasons):

--8<---------------cut here---------------start------------->8---
+ GUIX_BUILD_OPTIONS=--no-substitutes
+ export GUIX_BUILD_OPTIONS
+ guix pack --bootstrap guile-bootstrap
accepted connection from pid 25978, user marusich
The following derivations will be built:
   
/home/marusich/guix-wip-docker/test-tmp/store/5306akcwzxpl00jq237i5np684j0vmzl-tarball-pack.tar.gz.drv
   
/home/marusich/guix-wip-docker/test-tmp/store/cdvscfg50aw5jxnd259x1h9bp30lfkdj-module-import.drv
   
/home/marusich/guix-wip-docker/test-tmp/store/wr7q9d2fzkw1fi6q8ms8fkx9ivxjkqqg-module-import-compiled.drv
   
/home/marusich/guix-wip-docker/test-tmp/store/yirhq0bk23hs1ygx0b1ny3v56x9c9fji-tar-1.29.tar.xz.drv
   
/home/marusich/guix-wip-docker/test-tmp/store/fwlyz9ll798kzb509zklkqbgf6px4sj9-lzip-1.19.drv
   
/home/marusich/guix-wip-docker/test-tmp/store/dncnbczz63kq6zmdrr8vf5h5c2sb71pg-ed-1.14.2.drv
   
/home/marusich/guix-wip-docker/test-tmp/store/9fnrsfhfa4sh8fm86s7451gm9lma3ys1-pkg-config-0.29.2.drv
   
/home/marusich/guix-wip-docker/test-tmp/store/6vddknc0jvxl6nbqvimn6mdilvxavln4-m4-1.4.18.drv
   
/home/marusich/guix-wip-docker/test-tmp/store/ww1wddpxfj4ynbi3gspdnfpi1w6zh19l-expat-2.2.5.drv
   
/home/marusich/guix-wip-docker/test-tmp/store/77hapsriz2k7ix77gqgmsj5m2dxzshy3-gettext-minimal-0.19.8.1.drv
   
/home/marusich/guix-wip-docker/test-tmp/store/kprlkrdl1bmxghi1q53vm4bc3cxfiw61-attr-2.4.47.drv
   
/home/marusich/guix-wip-docker/test-tmp/store/93rzrq8dnl74rlfzvv1a4k1m553s7g57-libcap-2.25.drv
   
/home/marusich/guix-wip-docker/test-tmp/store/d5kc96m4why9yvnlvyal8qb5ar6gi7sw-perl-5.26.1.drv
   
/home/marusich/guix-wip-docker/test-tmp/store/w282an05cm2900wv8l7hk51rv1df3kl6-gmp-6.1.2.drv
   
/home/marusich/guix-wip-docker/test-tmp/store/xk9l4cnqqc0jq611vm7i5gm5f2zh505n-acl-2.2.52.drv
   
/home/marusich/guix-wip-docker/test-tmp/store/32m2qgabizs50hm70rjdc3zx4a64c1sx-libsigsegv-2.11.drv
   
/home/marusich/guix-wip-docker/test-tmp/store/0jv6207vr9d8dd1j4ipiqbvgsxxwb9ax-m4-1.4.18.drv
   
/home/marusich/guix-wip-docker/test-tmp/store/hdr6lkf7fwv7zgpryd9rnx2n9qqy017q-libatomic-ops-7.4.8.drv
   
/home/marusich/guix-wip-docker/test-tmp/store/15kqh766smp6jy64is5zy49yix20l4h9-libltdl-2.4.6.drv
   
/home/marusich/guix-wip-docker/test-tmp/store/2amv52bswrhvq82y6cx2dgbbaw1vnq32-libffi-3.2.1.drv
   
/home/marusich/guix-wip-docker/test-tmp/store/4niik1n2j3f62xv01x5h6jwmzm0f3nvz-libgc-7.6.0.drv
   
/home/marusich/guix-wip-docker/test-tmp/store/hy8c34s1q5a0af6jc61z4z3159c8gryq-gmp-6.1.2.drv
   
/home/marusich/guix-wip-docker/test-tmp/store/illsqfj4c1zdkszwyyw8b9r593lx8xvx-libunistring-0.9.8.drv
   
/home/marusich/guix-wip-docker/test-tmp/store/jz05x01mb6y7i3by3mwzj9bwqnmcvr6m-pkg-config-0.29.2.drv
   
/home/marusich/guix-wip-docker/test-tmp/store/rfcacc8gajq6sqvx5166cxal3qqjdv48-perl-boot0-5.26.1.drv
   
/home/marusich/guix-wip-docker/test-tmp/store/0sy6428wjldm79ksdmgif7ak6j63fsda-module-import.drv
   
/home/marusich/guix-wip-docker/test-tmp/store/rglsxbjr8aplljbl87ibqwq32f0zbfz5-module-import-compiled.drv
   
/home/marusich/guix-wip-docker/test-tmp/store/vgqjcfc86maww541pvkl8lzhbkayy427-gzip-1.8.drv
   
/home/marusich/guix-wip-docker/test-tmp/store/1malkdjyhma8payi1q39gpas819drs7m-module-import.drv
   
/home/marusich/guix-wip-docker/test-tmp/store/1n2b3c5drfdmc8ammc4fhhnz6wiqxlfg-tar-1.29.drv
   
/home/marusich/guix-wip-docker/test-tmp/store/3rm7z6iwjk36ccw2q2cchn5cy25gpbw9-glibc-utf8-locales-2.26.105-g0890d5379c.drv
   
/home/marusich/guix-wip-docker/test-tmp/store/9svg3a4sggjwjh7xjpra68mms3qrnzyy-bash-minimal-4.4.12.drv
   
/home/marusich/guix-wip-docker/test-tmp/store/9y3vbqgvzgf3z0a8wrh6n710m2nca3k1-gcc-5.5.0.drv
   
/home/marusich/guix-wip-docker/test-tmp/store/av9n8ds39jzxp13h6pzls4wlq62nd1ry-grep-3.1.drv
   
/home/marusich/guix-wip-docker/test-tmp/store/c1rm58rv7xyhkw5jqw0hljzdwkx87vv4-guile-2.2.3.drv
   
/home/marusich/guix-wip-docker/test-tmp/store/cc9nalqm463s7kwz9j093iaph45zmbm1-findutils-4.6.0.drv
   
/home/marusich/guix-wip-docker/test-tmp/store/dhz1a4ascsmfypkd1fl782fj1wn75k04-gzip-1.8.drv
   
/home/marusich/guix-wip-docker/test-tmp/store/gxgvm9d4x7n72pzfmr8bisq2j2jd5dw9-bzip2-1.0.6.drv
   
/home/marusich/guix-wip-docker/test-tmp/store/ixysd07gjzcfi3l7x61n7k4mbyi2mgzl-gawk-4.1.4.drv
   
/home/marusich/guix-wip-docker/test-tmp/store/plfrsghh3kk0kjjwsl9qh7mh4ppm4s7j-coreutils-8.28.drv
   
/home/marusich/guix-wip-docker/test-tmp/store/plsnrf4f6mfyw5270f3f46hfq9agikxi-make-4.2.1.drv
   
/home/marusich/guix-wip-docker/test-tmp/store/q6g6axx4i927mw8jx7kl8brjsxgkwgsp-ld-wrapper-0.drv
   
/home/marusich/guix-wip-docker/test-tmp/store/v97fmq5lbcawz4z3ljic971djdif5ayh-xz-5.2.3.drv
   
/home/marusich/guix-wip-docker/test-tmp/store/x34halk6yz9nmkd2wcipxgn6fmygdyn8-patch-2.7.5.drv
   
/home/marusich/guix-wip-docker/test-tmp/store/xa3a93dyfv3pkkg79jmaz37lb4p57s96-diffutils-3.6.drv
   
/home/marusich/guix-wip-docker/test-tmp/store/xyb0kwdmish8a299smx7hkq0fgrfn4qw-sed-4.4.drv
   
/home/marusich/guix-wip-docker/test-tmp/store/z0z9f2cxk2za96i8fczq23mh3h70d4n9-file-5.32.drv
   
/home/marusich/guix-wip-docker/test-tmp/store/z6ads1hjgrcj5jcaf7jinrh9bn2m7rmk-module-import-compiled.drv
   
/home/marusich/guix-wip-docker/test-tmp/store/gqb0hng3ndhgi8ddxnwn1lbi58vpbawl-gzip-1.8.drv
   
/home/marusich/guix-wip-docker/test-tmp/store/w9s9c0b8szkkx79qi910r7jjs1zwkzaw-tar-1.29.drv
--8<---------------cut here---------------end--------------->8---

It's quicker after the first time, but it's a hefty one-time cost.

I think so many things need to be built because "guix pack" creates the
pack using a gexp that uses packages.  For example, instead of executing
"tar" by looking it up in the current environment (e.g. via system*), it
builds the "tar" package and executes the resulting "tar" program in the
gexp that creates the pack.  Therefore, even though we do use
guile-bootstrap to run the gexp, Guix still need to build tar and all of
its dependencies, so the test takes hours to run.  Can we do better?

Maybe we could share the host's store with the test.  I don't think this
will help, though, since the package definitions used by the installed
Guix are often different than the package definitions in the Git
repository.  If we tried to do this, I think we'd probably just wind up
building tar etc. before running the test, instead of during the test.
The total time required to prepare and run the test probably wouldn't
change.

Maybe we could move tests like these to a "long-running" test target
(e.g., "make check-long-running").  I think this could work, but I would
much rather just write tests that run quickly.

Maybe we could rewrite "guix pack" so that it uses system* to run tar.
I think this might actually be a good idea, but I'm curious to know what
you think.  It's neat that one can write a program (like "guix pack")
whose runtime dependencies are built just-in-time by Guix.  However,
these runtime dependencies (e.g., tar) could theoretically be garbage
collected by Guix in-between invocations of "guix pack", and it clearly
makes the program a little more difficult to test.

Finally, maybe we could stick with running the tests in "dry-run" mode.
I'm in favor of this idea, but obviously I'm a little biased.  :-)

-- 
Chris

0001-tests-Add-tests-for-guix-pack.patch
Description: Text Data

signature.asc
Description: PGP signature