Re: Interoperation between package managers

[Stefan Monnier]

> - Is this use case documented in any way?

It's documented in elpa.git's README file.

>   I looked at the docstring for `package-directory-list' and it just
>   said that the variable is for system-wide use only,

It's a slightly crude way to describe it, indeed.  In reality the
difference between `package-directory-list' and `package-user-dir' is
that package.el will never write to `package-directory-list', it will
only add/remove packages from `package-user-dir' (so packages in
`package-directory-list' are assumed to be installed/removed some other
way).

>   Notably absent was any mention of what "directories containing Emacs
>   Lisp packages" means, since all sorts of different formats could
>   be expected.

The installed format of a package is very close to the ELPA format,
except that it additionally has a <pkg>-autoloads.el file (well, it's
also expected that it's been compiled, but that's not strictly
necessary).

It's probably not really documented, tho, indeed.

> - I had originally assumed that `package-directory-list' included
>   packages in an ELPA-server-compatible format, and that package.el
>   would display these packages in `package-list-packages', and you
>   could install them into ~/.emacs.d/elpa (thus making copies of the
>   files). You are saying this is not the case, right?

No, indeed, they're assumed to be installed just like those of
~/.emacs.d/elpa.  To control whether or not they're activated, you need
to use package-load-list.

> - Is this the intended way to use package.el? Every tutorial I've seen
>   only covers installing packages from a local or remote
>   ELPA repository.

It's the way I use it, so it's definitely a way I want to support.
It's not advertised very loudly, tho (I do mention it on this list
every once in a while, but IIUC very few other people use it this way).

My intention was/is to try and bring GNU ELPA packages closer to bundled
packages for Emacs maintainers:

    (cd emacs; git pull; make);
    (cd elpa; git pull; make externals; make)

updates both the bundled and the "unbundled" packages at the same time,
and both kinds are activated in my Emacs sessions.  And you get to see
the compilation warnings of both kinds of packages as well ;-)

> - When you modify a package, how is byte-compilation and autoload
>   generation handled? Do you just have to run 'make' again?

Yes.

>   Does that mean you have to hope that every package's Git repository
>   provides a byte-compilation and autoload generation mechanism, and
>   then remember how to use each of them? (Here I am talking about
>   packages that are not in GNU ELPA, i.e. the majority of them.)

This is not applicable to packages that are not in GNU ELPA.

This said, I also do the following:

   cd .../elisp; git clone .../otherpackage.git
   cd .../elpa/packages;
   ln -s .../elisp/otherpackage ./

so as to let "make" treat "otherpackage" as if it were part of elpa.git.
In most cases it mostly works (the autoloads generation mechanism as
well as the compilation rules are those of elpa/GNUmakefile rather than
those provided by "otherpackage") and depending on the changes needed
locally, I try to send them upstream or I keep them as local
modifications.  This is fine for my own use, but obviously not something
I'd recommend to the uninitiated.

> - Does package.el officially support installing packages from
>   version-control, or do you have to clone and manage the repositories
>   manually?

You have to clone&manage manually.

>   If not currently, is such support a future development
>   target for package.el?

Hasn't been so far.  Rather than integrate it into package.el, it could
be a completely separate package that helps automate what I do manually.
But for best results, it would likely benefit from some adjustments in
package.el.  I haven't thought about any of it.

Currently, the main way package.el was adjusted to help support the
"elpa.git checkout in package-directory-list" was to make it so that
packages in ~/.emacs.d/elpa don't have to use a subdirectory named
<pkg>-<vers> but it can be named just <pkg> (otherwise we'd need to
rename all those dirs after "git pull" to reflect the new version
numbers).

> So package.el interop may indeed be useful for Borg. It will not be
> useful for straight.el regardless, since straight.el makes
> compatibility with package.el an explicit non-goal. But in any case, I
> think the standardization you have proposed is a great idea, since it
> appears that every package manager other than straight.el could
> benefit from it.

As mentioned above, I haven't thought about what package.el could do to
make it easier to automate what I do manually, so if you have
suggestions for things that we could change in (or add to) package.el to
make it easier for Borg to interoperate with it, please send them along,


        Stefan