policy discussion on bundling ELPA packages in the emacs tarball

[Stephen Leake]

Attached is the file 'elpa', containing the current
emacs/admin/notes/elpa, plus a baseline policy for bundling GNU ELPA
packages in the emacs tarball, and some unresolved issues.

This is _not_ a final plan; it is a starting point for discussion.

(I tried to post it in the savannah ada-mode download area, but trying
to retrieve it gives me 404. And now I can't delete it. Sigh).

The biggest open issue is whether the bundled packages live only in the
elpa.git worktree, or if they are git submodules that are checked out into
the emacs.git worktree.

The baseline policy described in the attached file states that the
bundled packages live only in the elpa git worktree.

Eli has said he wants the bundled packages in the emacs worktree; I'd
like to explore the specific rationale for that.

For the specific example of ada-mode, I'm guessing nobody much noticed
when its files were removed from emacs/lisp/progmodes. Other packages
that are not used by many Emacs developers will be similarly not much
missed.

At the moment, ada-mode is the _only_ package we are proposing to
bundle, as a test case. But we need to plan for the future.

Currently, org and others are "duplicated" packages; they have files in
emacs.git master/lisp/[PKGNAME]/* and in elpa.git externals/[PKGNAME].
At some point, they could become "bundled" packages.

Consider what this would look like for org under the baseline policy.
The files in master/lisp/org/* are deleted, and "org elpa.git
externals/org" is added to the list of bundled packages used by 'make
dist' (I need to add explicit detail to that mechanism; Stefan Monnier
mentioned this might already be written somewhere).

Any Emacs developer that wants to continue using org has two choices;
install it via M-x list-packages, or, if they want the development head,
checkout ELPA, run 'make packages/org', and either install it from there
or add elpa/packages/org to their load-path. This is exactly what Emacs
and ELPA developers do now for non-bundled packages.

The baseline policy describes a shell script `checkout-bundled-elpa' and
an emacs command `add-bundled-elpa-packages' (for ~/.emacs) to simplify
using all bundled packages.

Now consider what bundling org looks like if we use some variant of the
"git submodules" policy. (for an introduction to git submodules, see
https://git-scm.com/book/en/v2/Git-Tools-Submodules). The files in
master/lisp/org/* are deleted, and "org elpa.git externals/org" is added
to the emacs/master/.gitmodules file (via 'git submodule add ...').
'make dist' includes emacs/master/elpa, so nothing in 'make dist' is
changed for 'org'.

In addition, Emacs developers who want the bundled ELPA packages would
run 'git submodule update --init'; this would checkout all the bundled
ELPA packages, into emacs/master/elpa/[PKGNAME]. Emacs startup code
would put these in the default load-path.

A fresh checkout of Emacs with the bundled submodules would be done by
'git clone .../emacs.git --recurse-submodules'. Ideally, the savannah
page describing how to get Emacs from git would say that. but many
people will forget the --recurse-submodules.


In both cases, the bundled packages are in load-path.

Finding them via C-x C-f (find-file) is slightly different; in the
baseline case it requires navigating to <root>/elpa/packages/... for
bundled packages. In the submodules case is requires navigating to
<root>/emacs/master/elpa/... Both are different from the current
<root>/emacs/master/lisp/..., so in either case you have to remember
which packages are bundled, and that can change over time. I suggest we
standardize on an emacs elisp project that allows using C-x p f
(project-find-file) to complete on files in load-path for elisp files;
that removes the issue of remembering where things are.

Another difference is testing for a release; that should always be done
with the identified release version of bundled packages. That is easier
to accomplish via git submodules, since they can identify a specific
version (I think?), or a specific release branch. Otherwise people doing
release testing would have to use `checkout-bundled-elpa' to get the
correct versions in elpa.

There will probably be a similar number of posts to emacs.devel about
"where is org?"; the answer is either "cd elpa; make packages/org" or
"git submodule update --init".

Comments?

--
-- Stephe

elpa
Description: Binary data