Upgrading packages with substitutes only (bug #26608)

[Timothy Sample]

Hi Guix,

I’ve been using GuixSD for a couple of months now, and it is super
cool. Thanks for all your hard work! There is one little thing,
though. :)

There’s a feature request (bug #26608) about adding an
“--only-substitutes” flag to “guix package -u”, which would only upgrade
packages that have substitutes. This sounds very useful to me, so I’ve
written code to implement it. It got a little harrier than I expected,
so I thought I would get some feedback before actually submitting a
patch.

Ludo suggested to check each package using the “substitute-paths” RPC
and filter them accordingly. I did exactly this, but there was a
problem. Packages were being built before my code even ran! It turns out
that calling “package-output” (or “package-derivation”) to get a
package’s output path may result in building the package. It’s the
grafting code that does this − it needs to get a package’s references,
and if the daemon gives it any trouble, it tells the daemon to just
build the package to figure it out. This is a real problem if you are
trying to avoid building. This all happens pretty early when running
“guix package -u”, since it compares output paths when figuring out what
needs upgrading.

Therefore, to make it work, I introduced a flag, “#:fail-on-build?”,
that I threaded through a few functions that lets me try to get a
package’s output path without resorting to building the package. The
change looks like this:

----------

diff --git a/guix/grafts.scm b/guix/grafts.scm
index d6b0e93e8..bcfc289a4 100644
--- a/guix/grafts.scm
+++ b/guix/grafts.scm
@@ -174,7 +174,7 @@ references.  Call REFERENCES to get the list of
references."
                  items))))
     (remove (cut member <> self) refs)))
 
-(define (references-oracle store drv)
+(define (references-oracle store drv fail-on-build?)
   "Return a one-argument procedure that, when passed the file name of
   DRV's
 outputs or their dependencies, returns the list of references of that
 item.
 Use either local info or substitute info; build DRV if no information
 is
@@ -186,6 +186,8 @@ available."
 
   (define (references* items)
     (guard (c ((nix-protocol-error? c)
+               (when fail-on-build?
+                 (raise c))
                ;; As a last resort, build DRV and query the references
                of the
                ;; build result.
 
@@ -298,7 +300,8 @@ derivations to the corresponding set of grafts."
                            #:key
                            (guile (%guile-for-build))
                            (outputs (derivation-output-names drv))
-                           (system (%current-system)))
+                           (system (%current-system))
+                           (fail-on-build? #f))
   "Apply GRAFTS to the OUTPUTS of DRV and all their dependencies,
   recursively.
 That is, if GRAFTS apply only indirectly to DRV, graft the dependencies
 of
 DRV, and graft DRV itself to refer to those grafted dependencies."
@@ -307,7 +310,7 @@ DRV, and graft DRV itself to refer to those grafted
dependencies."
   ;; upfront to have as much parallelism as possible when querying
   substitute
   ;; info or when building DRV.
   (define references
-    (references-oracle store drv))
+    (references-oracle store drv fail-on-build?))
 
   (match (run-with-state
              (cumulative-grafts store drv grafts references
diff --git a/guix/packages.scm b/guix/packages.scm
index f4967f98f..e792c348c 100644
--- a/guix/packages.scm
+++ b/guix/packages.scm
@@ -1123,7 +1123,9 @@ This is an internal procedure."
 
 (define* (package-derivation store package
                              #:optional (system (%current-system))
-                             #:key (graft? (%graft?)))
+                             #:key
+                             (graft? (%graft?))
+                             (fail-on-build? #f))
   "Return the <derivation> object of PACKAGE for SYSTEM."
 
   ;; Compute the derivation and cache the result.  Caching is important
@@ -1144,12 +1146,15 @@ This is an internal procedure."
                      ;; recurses anyway.
                      (graft-derivation store drv grafts
                                        #:system system
-                                       #:guile guile))))
+                                       #:guile guile
+                                       #:fail-on-build?
fail-on-build?))))
                 drv))))
 
 (define* (package-cross-derivation store package target
                                    #:optional (system
                                    (%current-system))
-                                   #:key (graft? (%graft?)))
+                                   #:key
+                                   (graft? (%graft?))
+                                   (fail-on-build? #f))
   "Cross-build PACKAGE for TARGET (a GNU triplet) from host SYSTEM (a
   Guix
 system identifying string)."
   (cached package (list system target graft?)
@@ -1164,7 +1169,8 @@ system identifying string)."
                                      #:system system
                                      #:guile
                                      (package-derivation store
                                      (default-guile)
-                                                         system
#:graft? #f))))
+                                                         system
#:graft? #f)
+                                     #:fail-on-build? fail-on-build?)))
                 drv))))
 
 (define* (package-output store package
diff --git a/guix/scripts/package.scm b/guix/scripts/package.scm
index a6bfb03ae..19b536dd5 100644

----------

If adding this extra parameter isn’t too ugly, then I will put together
two patches: one with this parameter, and another that uses it to
implement “--only-substitutes”. (And maybe a third that fixes a very
minor bug in “guix package”, but that’s another story.) I’m new to
Scheme programming, so this may be an atrocity of some sort that I just
don’t know about; go easy! :)


-- Tim