[Axiom-developer] CVS, Arch, Darcs

[Tim Daly]

I'm looking to the CVS, Arch, Darcs issue at the moment.
I'll try to either recreate Arch or piggyback on Darcs.

I have about a dozen or so "subprojects" of axiom I'm working on
and they are all in one local pile. I'd like to put them up on a
public server with several goals
(a) other people can hack them. Axiom has several areas that 
    need work, like the Mac OSX/BSD port and I want to be able
    to branch the code to handle these experimental changes
    until they get back to the main branch.
(b) I can get them cleaned up and straight. Given the rapid 
    task-switching I do during the day I'm usually too busy 
    to keep things clean. 
(c) keep a couple sub-projects "semi-private". some of the
    branches have papers that I don't yet have permission to
    reproduce.
(d) keep multiple projects. For instance, I'm hacking a large
    C++ code pile that will eventually work with Axiom but is
    really a standalone project.
(e) "pull through" another server. I want Doyen to be a superset
    of Axiom and MathAction so I want to make these projects
    sub-projects of a larger effort
(f) "pull from" another server. Axiom currently extends and patches
    GCL. I'd like to pull a gcl tarball transparently when Axiom is
    fetched.

so I want something like:

Doyen Scientific Platform
  ^
  | 
  (merge) Doyen involves many subprojects of which Axiom is one.
  |
  + (patches) ---(checkout)------------> Doyen+patches
              <--(commit)--------------- Doyen+patches
^
|
(pull thru) Doyen uses Axiom and should include it automatically
|
Axiom (main)  -------(checkout)------------> Axiom
              <------(commit)---------------
  ^
  |
  (merge) There are about a dozen sub-efforts
  |
  + (graphics) ------(checkout)------------> Axiom+graphics
               <-----(commit)--------------- Axiom+graphics
  ^
  |
  (merge)
  |
  + (hyperdoc) ------(checkout)------------> Axiom+hyperdoc
               <-----(commit)--------------- Axiom+hyperdoc
  ^
  | 
  (merge)
  |
  + (crystal)  ------(checkout)------------> Axiom+crystal
               <-----(commit)--------------- Axiom+crystal
    crystal needs to be private because I don't yet have permission
    to publish the documents/code involved
^
|
(pull from) GCL is required and I want to automatically pull a
|           particular tarball from the GCL tree when Axiom is fetched.
|
GCL
  ^
  | 
  (merge)
  |
  + (patches) ---(checkout)------------> GCL+patches
              <--(commit)--------------- GCL+patches
^
|
(pull from) Mathaction has it's own maintain tree but I'd eventually
|          want to use it as a new Axiom and Magnus common front end.
|
MathAction
  ^
  | 
  (merge)
  |
  + (patches) ---(checkout)------------> MathAction+patches
              <--(commit)--------------- MathAction+patches
^
|
(separate) Eventually these two will merge algorithmically but not now
           so Magnus would be a separate subproject
|
Magnus
  ^
  | 
  (merge)
  |
  + (cluster) ---(checkout)------------> Magnus+cluster
              <--(commit)--------------- Magnus+cluster


I'm sure none of this is quite clear and seems overly complex.
However, I'm now the lead on 3 major projects and work "next to"
and eventually with other projects. I do all of the above things
by hand at the moment and it would be very nice to have support
software capable of handling the job(s). 

Some of this is covered well by all of the software. I switched
from CVS to Arch and was working with it reasonably well, although
I got the impression that I "didn't get it" and was using it wrong.
I eventually ran out of disk space on the machine that was hosting
Arch (where I was a guest) so I had to take it down. 

I currently fund a machine for axiom development (axiom-developer.org)
out of my own pocket so disk space costs are less of a problem. However
my effort to get Arch working failed and I gave up in frustration. For
the life of me I can't seem to figure out how to get the web server to
give me the files. I did it once and can't get it to work again.

One problem I see is that the standard expectation is that every
project is "standalone". The user is expected to fetch and build
not only the desired project but also several other projects it uses.

For instance, Axiom uses GCL but it uses a patched version. I'd like
to be able to fetch the raw version from the GCL site or the patched
version "pulled thru" the Axiom site. I have to be able to pull a
particular version or tarball because I test the main branch of Axiom
every time with every change of GCL before I publish Axiom. You
can expect that the version of GCL used by Axiom will work (and you
can't expect that a non-tested version will work).

I no longer view Axiom as a "standalone" project and, as the
complexity of the pile builds up (Doyen will use Axiom, Magnus, and
MathAction just to name 3 projects) there needs to be support for
a single checkout point of a multi-project pile. The single checkout
point needs to present a "view" that includes patches to the subprojects.
(Subprojects won't always accept patches upstream). GCL faces the same
issues with things like gmp, etc.

I suspect Arch could be hacked so that it would invoke programs to
pull in "virtual branches" (like GCL) but this adds to the complexity
of an already complex program.

Really what we seem to need is a "make" style program that sits behind
the "checkout" request. A checkout for "axiom" would run a bunch
of stanzas to fetch, patch, and forward the correct, collected source
tree. CVS and Arch don't seem to have this functionality and even if
they did it would end up buried in a 300-option command line.

Methinks the world needs a new kind of person, a "project librarian",
who has the job of building and maintaining meta-projects by contacting
and working with other librarians. The complexity of maintaining a
repository that holds dozens of cooperating projects makes my hair hurt.

Anyway, one mistake at a time... I'll try to recover the Arch tree.

Tim