Re: What a modern collaboration toolkit looks like

[Alan Mackenzie]

Happy New Year, Eric and Emacs!

On Sun, Dec 30, 2007 at 07:22:17AM -0500, Eric S. Raymond wrote:

> ...., I think it will be useful if I start by giving everybody a clear
> idea of the potential benefits of changing our practices.

> I'm going to describe the collaboration toolkit on another project
> where I happen to be a senior dev, called Battle For Wesnoth.  You can
> find the project at <http://www.wesnoth.org/>.

> This is a typical modern open-source project.

Emacs is an atypical, very old, piece of free software.  Wesnoth seems to
be about 5 years old.  (I haven't found the repository online.)  This has
some bearing on the differences in development processes.

> .... Here are the collaborative tools we use every day:

> * Source control with Subversion
> * A bug tracker
> * A very active IRC channel used for routine dev chatter
> * A dev mailing list, used mostly for white papers and long proposals
> * Web forums where a large user community hangs out
> * Subversion commits are echoed to IRC in real time by a monitor bot
> * Subversion commits that reference a bug append a comment to the tracker
> * A bot on IRC that you can query with a bug number and get a tracker URL.

> Here are some of the ways my workflow on Wesnoth differs from my
> workflow on Emacs:

> 1. When I do a commit of changes to several files, the entire changeset
> is saved as a unit with a single change comment attached to it.  

I would appreciate having this in Emacs.

> a) That change can be referenced, through its Subversion revision ID.  
>    (So, for example, "Hey boucman, your r22615 broke linger mode!")

Hopefully, people are considerate enough not to do this too often;
continually having to look somewhere else to get context is not nice.

> b) That change can be backed out as a unit.

That's fine if nearly all changes are independent.  In Emacs, I think,
this is not the case.  The codebase is extremely old, and clean
structuring has in many cases been lost.  (But, hey, for >20 y.o. code,
it's not bad).

Such easy backing out could lead to problems.

[ .... ]

> 2. My commit is also echoed to the IRC channel, where there are almost
> never fewer than three or four devs hanging out as they hack, chatting
> in real time.  Review tends to happen instantly.

What about fixes that are too big for instant review?  Emacs has lots of
bugs (and often fixes ;-) that are barely tractable, so that the slower
pace of email seems a better fit.

[ .... ]

> The effect of all these tools is more than the sum of its parts.

> One huge one is simply that devs can choose to spend time picking bugs
> off the tracker and nailing them, with basically zero process friction.

> Another is that we routinely hash out serious design and coding issues
> in real time, because everyone on the chat channel can share hyperlinks
> to the codebase, the commit history, the bug tracker, and posts
> on the user forums.

I've no experience of IIRC.  But doesn't this way of working mean
continually dancing from one thing to another?  I don't do that very
well.  The "hyperlinks" bit sounds like you have to look at 3 or 4 things
simultaneously to be able to synthesise a context.  How does this working
style work for difficult problems?

> Yet a third is that when we decide to do it, we can converge on a
> releasable state with almost absurd ease.  Like, Ivanovic (our release
> manager) will announce "Point release scheduled this coming Wednesday"
> and everyone will pretty much flip into bug-stomping mode.  The tracker
> bug list tends to shrink dramatically when this happens -- not only do
> we get prepared for release but *we know we've done so*.

Eric, how well do you think this could work at all for Emacs?

> More generally, development happens *fast*.  I don't have to wait weeks
> to find out whether other devs think of an idea.  Commonly I know
> within minutes.

On emacs-devel, this takes hours, not weeks.  At any given time, most
Emacs developers are either asleep or doing other things (like earning
their living).  Doesn't this quick-fire style end up excluding lots of
hackers from participation?

> The Wesnoth devs are good but not exceptionally so, and we're weighed
> down by a crappy implementation language (C++).

Yuck!!

> Nevertheless our productivity, in terms of goals achieved per hour of
> work, is quite high.  That's because our collaboration tools support
> everything we do without imposing noticeable process friction.  This
> makes us dramatically more effective as individuals and as a group than
> we could possibly be without them.

Might it also be because your code base is still young, clean and
flexible?

> Lessons for the Emacs project?  Hell, yes.  But I'm not going to write
> that rant until y'all have had a little time to absorb this description
> of how things can be.

I'll look forward to that!  We'd all welcome a process for releasing
every few months rather than twice a decade.

-- 
Alan Mackenzie (Nuremberg, Germany).