Why Emacs needs a modern bug tracker

[Eric S. Raymond]

This is a slightly edited version of an argument I sent privately
to Stefan Monnier.  In view of Gianluca Della Vedova's observations
on this topic, posting it to the list seems merited.

Stefan wrote, describing Emacs's present workflow:
> In any case the "email access" [requirement] is mostly:
> - users should be able to report bugs via email (and via M-x
>   report-emacs-bug) rather than via the web.
> - all the discussion around a bug should happen via email, basically in
>   the emacs-devel mailing-list.
> 
> I.e. all we need in addition to what we currently have, is a central
> place to record "issues" and link them to specific threads of the
> mailing-list.  This central place should be easily accessible from
> within Emacs, of course, together with a few bindings to Gnus so that
> I can easily add the current message to the list of issues and so I can
> jump from an issue back to the relevant thread.

I think Stefan is expressing a view that is pretty common among the
old-school Emacs devs here.  Alas, I think it's quite far out of
touch with the reality of 2008.

For a project the size of Emacs even Stefan's "all we need" would
be seriously inadequate as described.  What he actually describe
having is so weak a support structure that it probably goes a long way
towards explaining Emacs's troubles and our pathologically long 
development cycle.

Let me explain how I know these things...

I lead another project called GPSD where we don't use an issue tracker
and don't miss it -- we do things the way Emacs now does, all email
bug reports all the time. One thing you can correctly deduce from this
is that I'm not simply attached to issue trackers in some habitual or
irrational way.  When they're not needed, I don't use them.

On the other hand, I know from observation that the Wesnoth project
is seriously slowed down and injured without its issue tracker.
The servers at gna.org occasionally flake out for an hour or two,
and I get to see what happens to productivity.  It's not pretty.

The difference is scale.  GPSD is about 60K lines of code, with a
COCOMO estimate of about 14.2 person-years and about 3 active
developers; normally we only have three or four issues active at once.
Wesnoth is a hair over 100K lines, with a COCOMO estimate of 25.6
years and 10-12 active developers; over the last year we've gone from
about 500 issues tracked to a bit over 300 (currently 196 feature requests,
89 bugs, and the remainder marked Fixed but not yet purged).

Somewhere in the scale range between GPSD and Wesnoth, having an issue
tracker moves from being a dispensable luxury to being an extremely
important tool. Actually, I'm pretty certain it's having a
well-structured issue *database* that's important -- the ability to do
queries like "show me all bugs on Mac OS/X assigned to edb" or "show
me all bugs of severity 'blocker' on the 1.3 branch".

One of the places a real issue database is most concretely useful is when
you're triaging bugs to close on a release.  It is *immensely* helpful
in making clear what needs to be done and at what point you are
finished doing it.

A sequential pile of bug emails, or even indexed email threads, simply
isn't a good enough organization for triaging or release prep at
Wesnoth's scale.  I don't think we'd be able to make four releases a
year if we were stuck to that, let alone a point release every three
weeks.

Now I consider Emacs: 1100K lines, a COCOMO estimate of over 328
years, and no issue database. I think I think I understand much better
now now why the team has only been able to ship one release in five years.
Trying to converge on a releasable state with as poor a view of the
Emacs bug load as we have must be damn near impossible.

Only...I suspect you (the team) don't know how poor your view is,
because you've never had a good view of a bug collection at your scale
to contrast it with.  (To be fair, it's only in the last two or three
years that I have grasped this myself.)

Beside the searchability problems with pile-of-emails organization
is another one; accepting emailed reports makes even *collecting* 
well-structured information about bugs highly problematic.  

The good thing about bug-tracker web forms from a developer point of
view isn't really that they're web, it's that they're *forms*. You can
channel your users into identifying the platform they're running on,
the preceived bug severity, and half a dozen other search keys.

In email, users will embed these things in running text and expect
you to parse them by eyeball.  Which means if you want a database
of bug reports instead of a mere pile of them, you get to re-enter 
each one.  Yuck.  In practice, nobody is ever willing to do this.  

This means that email bug entry needs to be deprecated, redirecting
people to a web form (or, in the special case of Emacs, a simulated
form in emacsbug).  Had you not wondered why Emacs is about the only
major project left that *isn't* trying to push its users onto an issue
tracker?  This is why...

To sum up the back-end argument, the reason I want Emacs to have a
real bug tracker is for the database it will build, so we'll have a
better view of our bugs.  From this (developer) point of view it's
accidental rather than essential that the all the good ones built in
the last decade are Web-facing.

Now let's talk about user experience.

*This* is where browser-centric interfaces become important.  I
know from recent experience on Wesnoth and elsewhere that most users
actually prefer using a well-designed web-based bug tracker to
emailing bug reports.  For them, it *is* about the browser UI.  They
find the structure and the sense of familiarity reassuring.

Gianluca makes exactly this point in his post.  Here is a further
reality check: My wife Cathy is an attorney, not a geek.  She cheerfully 
turns in bug reports on the Wesnoth tracker.  It is not even really
imaginable that she would email to a Wesnoth bug mailing list.  She
tried this once with the KDE guys and had a bad, *bad* experience
of geek snottiness.

Fortunately, most modern issue trackers allow you to feed all the
submissions and followon comments to a designated buglist.  So, if and
when we activate one, you old-schoolers will be able to keep your Gnus
readers.
-- 
                <a href="http://www.catb.org/~esr/";>Eric S. Raymond</a>

A ``decay in the social contract'' is detectable; there is a growing
feeling, particularly among middle-income taxpayers, that they are not
getting back, from society and government, their money's worth for
taxes paid. The tendency is for taxpayers to try to take more control
of their finances...    -- IRS Strategic Plan, (May 1984)