Re: [Gnu-arch-users] SCM vs CM

[Tom Lord]

    > From: "Neal D. Becker" <address@hidden>

    > There is a lot of momentum at my company to move to clearcase.
    > One of the arguments in favor of this is that clearcase offers a
    > complete configuration management (CM) solution, as opposed to
    > source code management (SCM).

    > It's not clear to me what that means.  What sorts of things are
    > needed for CM that are not addressed by a SCM tool, such as
    > gnu-arch?


The standard vocabulary in these areas is not very standardized, as
nearly as I can tell.

"Revision Control" refers to the archival, cataloging, manipulation,
and coordination of changes made to a collection of source files.

"Configuration Management" refers to the controlled combination and
transformation of collections of source, the tools used to transform
them, and the external resources combined with them in order to
produce products and/or deployed environments.

"Source Code Management" is, in my view, a blanket term that includes
revision control, configuration management, and other tools,
techniques, and activities specifically concerned with manipulating
source in a controlled manner.   (As an example, `inventory', viewed
as separable from `arch', is neither revision control nor
configuration management -- but it is a source code management tool.)

In my view, the features of CC that qualify as "configuration
management" are those that integrate the build process with the
revision control system.   Paraphrasing their product literature,
these are features like "Version all assets" (meaning, version build
products such as object files and libraries) and "Ensure build
accuracy" (meaning to audit and record all versioned dependencies that
contribute to a particular build -- ensure that the build process
regards the state of the revision control database as the definitive
statement of "what has changed").

Arch doesn't provide quite that whole range of features and isn't
intended to.  The `config' mechanism in arch provides a convenient
mechanism for recording and managing "slices" through the available
source resources in terms of revision control coordinates.  That, plus
the revision namespace, should be the point of interface between
revision control and the larger range of configuration management
problems (i.e., arch does its part in this area and then stops there).
I don't think that the trade-offs of versioning build intermediates
and hardwiring build auditting into the system in the manner of CC are
worth it -- I'd solve those problems differently in an arch shop.

I am a bit skeptical that it is the configuration management features
of CC, in the sense just described, that are why CC has momentum in
your shop.   Perhaps it is, but:

My guess would be that the big selling points are:

1) Prior CC experience among some of the principles or their friends.

2) The IBM Rational family of products goes well beyond just revision
   control and configuration management:  it's closer to being a 
   "director of engineering in a box".    That is, the products 
   are designed around the idea of telling you, more or less, how to
   run a software project:  the "UCM process" stuff -- divide your
   projects up into activities;  structure your branches by these
   activities; give your developers the ability to say "Ok, I'm
   working on activity FOO -- give me a work environment with the
   appropriate sources".  (Broadly speaking, I think arch needs
   something along the same lines -- that's the idea behind "overarch"
   that you might hear mentioned here from time to time.)

3) CC may have an expensive license, but there's a trade-off there.
   In return, you can hire from a competitive market of people who put
   "CC administrator" on their resume.   You can enroll your
   programmers in a training course to learn the processes CC
   encourages.   Looking at their product literature, you can even
   hire IBM Rational to come in and assess how good a job you are
   doing using their products and make recommendations about what
   you should do differently

I think that for software product managers, there's a kind of folklore
about two particular kinds of common disaster that points (2) and (3)
address and thus why point (1) tends to remain important:

The two disasters might be called:

* The development chaos scenario:  You decide just to trust your
  development team to "self organize" and work out whatever
  engineering processes and infrastructures they are most comfortable
  with.   Subsequently, they (a) spend _way_ too much time fussing
  over those issues (without converging on a stable solution) and your 
  key programmers aren't actually making product;  (b) they get it
  wrong and you discover, say, 2 weeks before you're supposed to ship,
  that nobody has ever actually built the product from anything other
  than their ad hoc private collection of sources, that you have
  unintegrated changes all over the shop, and that it's going to take
  weeks or months to actually pull a product out of the tangle.

* The loss of assets scenario:  Oops, the guy you fired trashed the
  source;  or the disk crash and it turns out the backup scripts had a
  bug;  or the NFS bug zeroed out a dozen blocks in your CVS files 
  and you lost 3 weeks of work.

Accurately or not (and it doesn't seem, from over here, all that
inaccurate) I think a widespread perception is that by throwing a
bunch of money at Rational and setting up shop according to their
recommendations you can largely avoid those disaster scenarios.
It's "just" a "one less thing to worry about" kind of selling point.

There are trade-offs to that.   Products such as CC seem to me to
really strongly encourage a particular style of programmer
organization and to put some pretty interesting constraints on how you
set up your environment.   For some styles of development, no doubt,
that's just fine -- appropriate even.   

Over the past decade, though, we've seen the emergence and gradual
refinement of "open source development processes", often interfacing
in important ways not just remote sites and partners, but also often
competitors and the "public at large".  Among the interesting
characteristics of these processes is the importance of _unplanned_,
opportunistic, development, often by 3rd parties with whom you have no
business relationship.  Inflexibly structured engineering processes,
complex administration requirements, and expensive license fees (and
for that matter, non-free tools) are not well suited for this world.
To be sure, the need for orderly, controlled product development is,
if anything, more pressing in this world than in the old world -- but
the form of that order and control has to be quite different.

So, where does GNU arch fit in when viewed as a CC competitor?

Well, for small shops for which the licensing fees are an issues,
clearly arch is one of the viable alternatives.

Featurewise, GNU arch is lacking high-level tools for formalizing
engineering process.   Some issues that need to be addressed: better
integration with bug trackers and planning tools;  tools for
patch-queue management;  and `overarch' -- a scripting tool with which
projects can express the "rules" for how new developers should
instantiate a development environment and join the fun.

On the configuration management front: the `configs' mechanism in arch
is a decent but small start -- it could certainly use some fleshing
out.   I do not think that it would be wise to try to extend arch with
features for build auditting and versioned build products:  that's the
wrong approach for today's environments.   Rather, I expect to see
continued development of orthogonal tools, each of which addresses a
little bit of those problems in an independent way, and which will
eventually be collectively integrated in a nice way.   There are
plenty of bits and pieces floating around already and already you can
do a decent enough job of "faking" these features, as for example the
way we use the Savannah bug-tracker.

On the ancillary merchandise front: there aren't arch training
courses, yet, and there isn't a competitive market of "arch
administrators".  It's a bit hard to bootstrap such things.  Perhaps
it will start to happen in a grass-roots way, as independent
contractors deploy arch for their customers and start to develop
themselves in those roles that way.  Perhaps it will start to happen
in a "kick-start" way -- if one or two large projects commit to using
arch and, in the course of that, essentially funding the development
of training techniques and formalized administrative "best practice"
recommendations.  We'll see.

-t