Re: Adding async package to GNU ELPA (was: [elpa] master 88578a4: Increase the default number of hits)

[Artur Malabarba]

> I must confess I have no idea how to add the package to ELPA. I suppose
> it shall be an external package.
>
> Could somebody who knows do it? Or tell me where I could get the needed
> information how-to?

See the README file in elpa.git. I've pasted the file below for your
convenience.
The "External branches" section explains how to use subtrees or a
separate branch.


-----
Copyright (C) 2010-2011, 2014, 2015 Free Software Foundation, Inc.
See the end of the file for license conditions.


This branch contains the sources, deployment scripts, and auxiliary
files for the Emacs Lisp package archive (elpa.gnu.org).

This file explains the branch layout, how to add and edit packages,
and how to deploy the archive (either on elpa.gnu.org, or a local copy
for testing purposes).


* DIRECTORY LAYOUT

** admin/    -- scripts for administering the package archive.
** html/     -- HTML for the elpa.gnu.org website.
** packages/ -- source code for the packages.


* PACKAGES

** Contents of the packages/ directory:
This directory holds the package sources, with one subdirectory for
each package.

Each directory in here corresponds to a package, which can be
either a single-file package or a multifile package.

A nightly cron job refreshes the GNU ELPA archive from this repository.

This cron job only creates a new package when the "version" (as specified in
the "Version:" header) of a package is modified.  This means that you can
safely work on the next version here without worrying about the unstable
code making it to GNU ELPA, and simply update the "version" when you want to
release the new code.

** To add a package: (submission, submit)

Adding a basic package is very simple. There are thorough
instructional, but the gist is that you:

1. Notify address@hidden
2. Place all files inside `packages/<pkg-name>/'.
3. `git add', `git commit' and `git push'.

If you don't have push access to the repository, someone will do steps
2 and 3 for you.

*** Notify address@hidden

There is no approval process for GNU Elpa packages.  Still,
you must send an email to emacs-devel for several reasons:

- Notifying other developers;
- Making sure the package doesn't break FSF rules;
- Checking if the package is not reinventing the wheel;
- Ensuring that first-time developers are doing it right.

Before doing anything, please ensure your package follows the
conventions described in the `** Format' section.  Then, send an email
to the list with the subject:
    [ELPA] New package: <pkg-name>

Start your message with an explanation about the package.  A
copy-paste of the package's Summary and Commentary is perfectly fine
here, but you can write more or less than that if you'd like.

At the bottom of the message contents include the changes you're going
to make (the patch).  For a single-file package this can be the
package file itself instead of the patch.  If you prefer (and if you
have push access), you can push your changes to a branch called
`scratch/<pkg-name>', and mention the branch in your message.

After 48h, or once any issues have been addressed, someone will push
your changes for you.  You should probably also subscribe to
address@hidden, since that's where we discuss about GNU Elpa, and
to address@hidden, since that's where people will report bugs
about your package.

*** Add a simple (1-file) package as packages/<pkg-name>/<pkg-name>.el.

The file needs to follow the usual coding conventions (most
importantly start with ";;; <file> --- <description>") and have a
"Version:" and "Maintainer:" pseudo-header (see the "Format"
subsection below).

For some examples, see
    (info "(elisp) Simple Packages")

*** Add a multi-file package as a directory, packages/<pkg-name>.

It needs to have a file named packages/<pkg-name>/<pkg-name>.el which
follows the
same rules as above.

It additionally follows the same guidelines described in
    (info "(elisp) Multi-file Packages")
with the exception that it is not a tar package (it's a plain
directory) and it must not contain a "<pkg-name>-pkg.el" file (this
will be created for you).

*** Commit your changes the usual way ("git add", "git commit", etc).

Changes in the Git repository do not immediately propagate to the
user-facing archive (what users see when they do `M-x list-packages').
That is done by deploying the archive, which happens automatically
once a day, and the changes are only reflected when the "Version:"
header changes.

** Format

Each package should follow the ELPA packaging conventions, but there are
some differences due to the way the deployment script creates the packages
and the web-pages from this source code:
- Multi-file packages put the package metadata in the main <pkg-name>.el file
  in the format used for single-file packages: the <pkg-name>-pkg.el file is
  auto-generated from it.
- Every package should have both a "Version:" *and* a "Maintainer:".
- the "URL:" header can be used to specify the home page
  of the package, if it's maintained externally.
- A "News:" section (or "NEWS" file) can/should be used to list the
  user-visible changes of each version.
- The "Package-Type:" header can be used to force the type of package
  created (can be either `simple' for single-file packages or `multi' for
  tarballs).  By default the type is decided based on whether there are
  several Elisp files in the source.
- If you want some files to not be included in the tarball, you can
  put a `.elpaignore' file in the root of your package directory, where you
  can list patterns of files to ignore (this file is passed to tar's -X).

** External branches

The above instructions are enough to add regular packages, those that
are maintained primarily here in the repository.  The instructions
below are for those maintainers who prefer to use a dedicated
repository or branch for the package.

There are two ways to do that: subtrees and externals.

Either way, such packages should always be listed in the
`externals-list' file.

In both cases, a copy of the code is kept in the `elpa' repository
(not necessarily in the master branch) and should be sync'd with the
upstream every once in a while.  This copy may include local changes,
although these should be kept to a minimum.

If know you don't want a local package, but don't know which of these
two options you prefer, then use a subtree.

*** Subtrees

In the `subtree' case, the copy of the code is kept here in the master
branch, inside its corresponding `packages/<pkg-name>' directory just
as if it were a local package.

In fact, a subtree package is essentially indistinguishable from a
local package.  The only difference is that, instead of developing it
here, you do it in some remote repository and pull in the changes.

Instead of manually creating the directory, you should be able to use:

    git subtree add --prefix=packages/<pkg-name> <remote-repo> <remote-branch>

Later, when you make some changes to the remote and want to publish
them here, simply do:

    git subtree pull --prefix=packages/<pkg-name> <remote-repo> <remote-branch>

On older git versions "git subtree" might not be available.  You can
try "git merge -s subtree", or just update git.

- <remote-repo> is the remote's URL.  If you've previously used "git
  remote add", then this can be the remote's name.
- <remote-branch> is the branch you want to pull (probably "master").

If you want the local code to be slightly different from the remote,
simply commit further changes to it here.  Of course, this may trigger
merge conflicts when you do a "subtree pull" in the future, so it's
best to avoid these local changes.

If someone makes changes to your package here on elpa.git and you want
to push them to your remote, it's easiest to just copy these changes
over to the remote repo.  Trying to push a subtree with git is likely
to induce headache.

**** When you're adding and pulling, DO NOT --SQUASH!!

Don't worry about flooding elpa.git's commit log with your package's
commit messages.  Your package is part of elpa.git.  Squashing doesn't
help and only gets in the way.

*** Externals

In the `external' case, the copy of the code is not kept here but in the
`externals/<pkg-name>' branch in the `elpa' repository.

You can check out all the external packages into the `packages' directory
with the command:

   make externals

You can check out a specific external PACKAGE into the `packages'
directory with these commands:

   cd packages
   git clone --reference .. --single-branch --branch externals/PACKAGE
$(git config remote.origin.url) PACKAGE

If you already have a packages/PACKAGE directory with a previous
checkout, you can update it like this:

   cd packages/PACKAGE
   git pull

** Public incubation

If you want to develop a package publicly prior to its first release (to
benefit from others' feedback, primarily), but not in an external repo,
you have 2 choices:
- you can simply put "Version: 0" to indicate that this should not be
  released yet.
- or you can push to an "ephemeral" branch -- subject to rebase and eventual
  removal upon finishing merge -- for the duration of the incubation.

* DEPLOYMENT

** To install all the packages "in place":

   make externals
   make

This compiles and generates autoloads for all the packages in the
packages/ directory.  You can then add that directory, e.g. with:

(eval-after-load 'package
  '(add-to-list 'package-directory-list ".../elpa/packages"))

** To deploy the package repository as a remotely-accessible archive:

   git clone .../elpa
   (cd elpa; git clone .../emacs)    #If you want to generate :core packages.
   mkdir build
   cd build
   (cd ../elpa; git log --format=%H | tail -n 1) >.changelog-witness
   ln -s ../elpa/admin
   ln -s ../elpa/GNUmakefile
   admin/update-archive.sh

This deploys the packages to the staging/ directory (sibling of "build").
Unlike "make", this makes a full copy of the packages, tars up
multi-file packages, and doesn't byte-compile any files.

** To access a deployed archive

To access the archive via HTTP, have a symlink (say) /var/www/packages
pointing to DEST/packages, and set up Emacs with

  (setq package-archives '(("new-elpa" . "http://foo.com/packages";)))

You can also access the archive via normal file access.  Such "local
archives" are useful for debugging:

  (setq package-archives '(("local-elpa" . ".../elpa/packages")))

** Notes specific to elpa.gnu.org

The way things are set up on this machine, we refresh the archive by
a cron job.  You can do it by hand by logging in (access set up by FSF
admins), and

   su elpa
   cd ~elpa/build
   admin/update-archive.sh

Which makes a full archive deployment, as discussed above.  The symlink
/var/www/packages points to the staging package directory under
/home/elpa/.

The Org mode dailies are also fetched and added by the script
admin/org-synch.sh, run as a cron job.

This file is part of GNU Emacs.

GNU Emacs is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.

GNU Emacs is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

You should have received a copy of the GNU General Public License
along with GNU Emacs.  If not, see <http://www.gnu.org/licenses/>.

Local variables:
mode: outline
paragraph-separate: "[ ]*$"
end: