Re: Python and propagation

[Christopher Allan Webber]

Ricardo Wurmus writes:

> 1) print a warning when a collision is expected to happen, not when a
> collision has happened.
>
> Guix knows what is currently installed in a profile, so it knows that,
> say, “python2-numpy” is installed.  It also knows that installing
> “htseq” is also going to install “python2-numpy” into that profile.  It
> also knows that the propagated “python2-numpy” is different from the
> installed “python2-numpy”.  Knowing all that, it should also know that
> there are going to be file conflicts, no?
>
> If that’s all true, could we make Guix print a warning about impending
> doom *before* it creates a new profile generation?  It’s a much nicer
> user experience (at least to me) when I’m being told about possible
> conflicts (at the package level) before Guix creates a new profile and
> nonchalantly informs me about arbitrarily resolving conflicts (at the
> file level).  The difference is in the amount of warnings I get (package
> vs individual files) and about the time I would otherwise have to waste
> to roll back the profile and upgrade already installed libraries or
> choose to install “htseq” into a new profile.

This would be nice, or at least, is there some kind of
"guix package --show-collisions" or something?

> 2) avoid PYTHONPATH, patch all Python files invasively!
>
> Python does not have any feature that is comparable to RUNPATH.  It is
> only concerned with finding libraries/modules by *name* in one of the
> directories specified by the PYTHONPATH environment variable.
>
> But actually the PYTHONPATH variable is not the only means to affect the
> search path for modules.  It is possible to change the search path
> programmatically:
>
>     import sys
>     sys.path.append("/gnu/store/cabba9e...-numpy.../lib/...")
>     import numpy
>
> The first two lines add an explicit store item path to the search path;
> the third line just imports the numpy module by name (as usual).  Even
> without setting the PYTHONPATH to include the numpy in the profile the
> third line won’t fail.
>
> I wonder if we could design a phase that — very much like the
> “wrap-program” phase — modifies *every* Python file and injects lines
> like the first two in the example above, appending explicit store item
> paths, so that all dependent Python libraries can be found without the
> need to have them installed in the same profile and without the need to
> set PYTHONPATH.
>
> Maybe this is crazy, and maybe this causes other annoying problems, but
> I think it is the closest we can get to a RUNPATH-like feature in
> Python.
>
> What do you think?  Do I need a long vacation or is this something we
> might realistically do in an automated fashion?
>
> ~~ Ricardo

So... that may work!  Except... well, I've raised this on another
thread, but I worry about late-binding based plugins or programs that
themselves call other python scripts (same with Guile, etc).  The
previous example I gave was, what if I write a Haunt extension library,
and I need to call "haunt.scm" and pull in that stuff?

  http://lists.gnu.org/archive/html/help-guix/2016-02/msg00032.html

What also about MediaGoblin plugins?  MediaGoblin allows you to define
plugins for different media types, and etc.  If you're adding a
"gmg_frobnicator" plugin, can MediaGoblin find gmg_frobnicator on its
PYTHONPATH when run?  And the reverse question: can gmg_frobnicator
import back into MediaGoblin and its dependencies?

I'm not sure...