Project systems (again)

[Daniel Colascione]

I'd like to include a good, minimal "project system" in Emacs 24.5 and
turn it on by default.

A "project system" is a set of commands and infrastructure functions for
finding and operating on groups of related files. Some features we want
to provide are running compile and shell commands on the project root
(as opposed to the directory in which a particular file lives), using
completion of easily locate a file in a project, and performing vc
operations on the project as a whole.

We should ship a minimal core and leave tasks like aggressive caching,
build system generation, customize-style project changes, source
indexing, and so on to other packages. That said, the core should
provide a way for different modules that provide "project" functionality
to talk to each other. Users should be able to easily customize how the
system works, or turn it off completely.

Implementation-wise, each project should be a lisp object we store in a
`project' buffer-local variable. When we first visit a file, we'll run a
hook to find the right project object, then store it in `project'.  (A
bit like `vc-handled-backends').

Each project object could just be a simple lisp closure that accepts the
function itself, a request symbol and a variable number of arguments. If
a project function doesn't know how to handle a particular request, it
should forward all its arguments to another project function or to a
default project function (say, `project-perform-default') that we
provide. This way, we allow simple kinds of project reuse and support
ad-hoc extensibility without the heft and conceptual complexity of an
object system.

If we visit two files in the same project, it might be useful for both
buffers to have the same value of `project'. We can make that work, but
when the last buffer that refers to a particular project is killed, we
should forget the project object entirely. Accumulating state as Emacs
opens and closes many projects makes it hard to reason about the system.

At the infrastructure level, project objects provide services. The two
services that interest me are requesting paths of a particular kind in
relation to a particular file and requesting a single path in relation
to a particular file.

The first might be something like this: project-paths PROJECT &key FILE
KIND. `project-paths' would return a completion table. We'd define some
well-known KIND values --- e.g., 'source-files, 'all-files,
'include-paths --- and let individual project implementations figure out
how to provide this service. For example, semantic might want to know
all the C include paths for a file, so it would call `project-paths'
with FILE being the name of the file it's about to parse and KIND being
'c-include-paths. An automake project could inspect generated makefiles
to determine the correct include paths for a given file. If a project
doesn't understand a KIND, it should just pass the request to its parent
project or to `project-perform-default'.

In stock project types, we should provide 'all-files and 'source-files
KIND values, making a good guess for the latter. Most of the time, we
can use vc to very quickly find the list of files --- just like
projectile does.

We should also provide something like this: project-path PROJECT &key
FILE KIND. This function should return a single path, or signal if we
can't find that path. This facility makes it easy to build "just give me
the damn project root" functionality, sure, but it also lets us provide
other kinds of relevant paths. (Imagine KIND being 'generated-file, say.)

Projects should also provide a user-accessible general-purpose plist,
since these things come in handy in all sorts of situations.

We also need to provide a good user interface. We should clone a subset
of Projectile's interface --- except that we'd want to use a
non-user-reserved prefix (say, C-x ,) instead of C-c p. In particular,
we need commands for finding a file in a project (with completion --- so
if you were working in Emacs, you'd be able to just type "lisp.h"),
running commands in the project root, running find-dired, and so on.

We should avoid aggressive caching in the project system. Modern systems
are very fast and we shouldn't complicate the implementation (and
debugging) for the sake of corner cases.

Performance should be fine. In the common case, the runtime overhead
will be a few additional calls to locate-dominating-file --- beyond
these calls, you only pay for what you use. The most expensive operation
is providing a completion table of all the files in a project: vc should
be able to make this operation very fast in practice, since vc-backed
projects are very common these days. We can also refer to GNU global,
cscope, and so on files as available.

EDE already provides some of the functionality I've described above, but
I don't like the way it does it. It's very complicated and embeds
uncommon features into the core. EDE's reliance on EIEIO and its large
feature-set create difficulties dumping the system with Emacs. And have
you tried actually creating a new EDE project type? It's surprisingly
and disappointingly difficult. Ad-hoc extensibility with EDE is hard
because of its EIEIO use, and I don't think EIEIO is buying us anything,
really.

EDE also includes many features that are not generally useful (like
makefile generation) and that complicate the codebase. EDE being
developed out-of-tree makes it hard to fix these issues. I'd rather have
a minimal and more idiomatically Emacs-ish core facility that EDE can
build on. EDE is also not currently integrated with VC, and it doesn't
provide nice user interfaces (ede-find-file, for example, doesn't do
completion.)

Projectile (https://github.com/bbatsov/projectile) is another popular
project system; its design is a bit closer to what I describe above.
It's not included in Emacs though, and it's not factored into an
extensible core like I described above. (In particular, it's written in
terms of project root files rather than abstract project-finding
functions in a function list.) I also don't know whether the author
would be interested in contributing the code to Emacs.

signature.asc
Description: OpenPGP digital signature