A "busy beaver" approach to assisting package definition updates

[Christopher Allan Webber]

Hello!

David Thompson and I had the good fortune to be able to hang out with
the esteemed Gerald Sussman in his office over tea.  Topics were wide
ranging, but one in particular has stuck in my mind as to how it could
apply to Guix.

What Sussman said is he thought version skew was a source of many
problems in practice (something I'm sure most of the Guix community
agrees with, and Guix avoids it somewhat by being much more precise).
Sussman also suggested something that I think he was thinking about in
terms of more distributed systems (but I'm not really sure); I'll give
an example of how it might apply at the end of the email since it's
maybe not applicable to Guix (but could be to an actor model or
propagator system).

Sussman's suggestions is that programs themselves should be able to
reach out and test to see if they can operate with the newer version of
some library and make the decision of which one they can link to.  And
they should be able to tell by having a reasonable test suite, and
checking to see if their test suite works once the linking happens.
It's not a perfect check of course, since tests aren't perfect, but it
would give programs more autonomy in linking by "exploring" their
surroundings.

Now obviously in Guix, we're trying to be very specific about linking,
and it's done at package build time, not at compile time.  But still, we
can mostly pull off the same thing using a continuous integration
system, assuming we have enough resources; we already have tooling to
look for the newest versions of packages, and we could possibly have a
CI system try to write out a new package definition based on the latest
version, try compiling it and see if it passes its tests... and then we
can try to see if the packages that depend on it can also compile
against it and pass their tests.  In this way we could automatically do
the work of detecting and suggesting upgrades for many packages, and
even finding out what they would break... *before* community members
have to spend the effort to try it themselves.  It wouldn't be perfect,
but it could be a big assist.  Perhaps there could be a web interface
that shows "Here's a new definition of this package, it seems to build.
Here's packages that depend on it which pass... these ones don't,
perhaps they should be pinned to the old version or be investigated?"

Of course, this would require a lot more build resources than the
already-strapped amount we have.  (If we had Guix running on "commodity
hosting" platforms, I suspect we could get some of these resources
donated.)

Helpful?  Not helpful?  Maybe it's obvious to everyone already, I dunno :)
 - Chris

BONUS CONTENT!  How would this apply to a distributed system, such as an
actor model (or propagator system or something), which might run into
other actors that it has to choose whether or not to link up to in real
time?  Actors in the system which become aware of other actors they
might interface with could either "test" linking up with the other
actors by running their tests (assuming they can interface
non-destructively) and also perhaps if there's a way to get some kind of
signature of what this actor's version or something identify'y looks
like, even if things go badly they could record information on whether
or not to link up to it in the future.