Re: [Axiom-developer] Terms of Surrender (was: BAD tim)

[William Sit]

Dear Bill:

I have been staying on the sideline (since I don't know much about Boot or Lisp)
on this "war", but I am more convinced by your arguments than by Tim's or
Morrison's, and I felt that I should be a late ally before you surrender!

First about Boot. My (admittedly poor) understanding of Boot is that it is an
intermedate language between Lisp and Spad (and there are other intermediate
languages Shoe, Meta, etc as well). I agree that to build a truely high level
language like Spad/Axiom/Aldor, intermediate languages are necessities and not
evils. It's much like what a mathematician does all the time (and most original
developers are mathematicians, in addition to being computer scientists): if the
proof of a theorem is too complicated, such as that for Fermat's Last Theorem,
you build up the proof in steps providing tools along the way. Even for simpler
results, there are usually lemmas, propositions, new definitions to abbreviate
frequently used concepts, new results based on definitions, etc. before the
final proof. A theorem proofer is also built that way. No one wants to reduce
every proof to its most fundamental logic. Whitehead and Russell did that in
2000 pages of Principia Mathematica, but how many people read it? Even there,
they "introduce a stratification of Frege's formulas into types, and to use this
to restrict which of Frege's formulas would be permitted in their logic. The key
idea was that a formula could not be substituted for a variable x in a formula
unless the variable x was of the appropriate type." (Quoted from a brief
synopsis in http://www.math.uwaterloo.ca/~snburris/htdocs/WWW/PDF/principia.pdf)
That sounds much like using Boot to restrict which constructs of Lisp would be
permitted to support the strong typing in Axiom. (OK, I am stretching the
analogy, but the parallel is irresistable.)
The writer of that quoted article also noted: "they quickly adopted the
convention of leaving out easy steps of proofs".

Second, I think once the bootstrap step is done, it is ok to write a compiler
for the language in the language itself. So writing a compiler for C in C, a
precompiler for C++ in C, one for Boot in Boot, or for Axiom in Axiom is
perfectly reasonable. The purist may want to boil everything down to
hand-crafted assembly (dare I say machine?) language, but honestly, while there
is a role to play when frequently called routines need to be inlined for
efficiency to use a lower level language, the majority of the code can and
should be replaced by as high a level language as usable. Such a stratification
allows easier understanding of the entire build and is a compromise and probably
an optimized compromise, when we consider all factors about efficiencies
(including those of developers and users, not just code). In any case, whether
eventually the project decides to turn everything to Lisp or to retain but
streamlining the intermediate languages, the most important step is documenting
the code in these intermediate languages. I believe when this is done, there
will be no need to reduce everything to Lisp; this does not mean there are no
opportunities to improve efficiency of code execution for some frequently used
Boot code by coding them in Lisp or even assembly languange. Rather, these
opportunities will be identified by the documentation and code analysis phase
(much like Knuth's MIX assembler, which keeps count of the number of times any
line of code has been executed).

Third, as far as Axiom code is concerned, and here I may claim some familiarity,
there is almost no need for a user (mathematician or scientist or student) of
symbolic computation to know (1) the architecture of the system (2) the various
intermediate languages (3) the algorithms behind the functions (4) the mechanics
of user interfaces, and more. All they need is a consistent syntax, well
documented, and a reliable answer returned each time. When a symbolic
computation system approaches the ease of use as a scientific calculator, that's
when it is most useful.

Fourth, for a developer of algebra code, it will help if (s)he understands some
basic design philosophy behind the language, but that is not necessary. If the
syntax and documentation are clear, and if the developer knows the subject
matter (mathematics, physics, etc) and the algorithm, (s)he can pick up the Spad
language easily.  History supports this observation: many of the Axiom algebra
packages beyond the basic categories and polynomial algebra were developed by
visitors to IBM (I was a visitor in 1988, after I was introduced to Scratchpad
by another visitor, Fritz Schwarz, who co-developed with Bronstein the LODOF
pacakge). Quoting myself from another post:
"When I first began coding in Axiom (the project was to compute first integrals
of dynamical systems, which was completed in less than a year in 1988 [snipped]
... I knew nothing about the design philosophy of Spad, nothing of Boot or Lisp,
didn't know how to use Boot to debug erroneous code. There was no documentation
(the book was out only in 1992). I did have help from the original developers
and I had to learn Groebner basis along the way. So all it really took was some
guts to go ahead." 
Even though the compiler often gave (and still does) poor or meaningless error
messages, it was always possible to find my own mistakes by the logic of the
program and a few print statements inserted at strategic places. But these are
just typical programming skills.

Scott Morrison wrote:
> Being one of those 10 people, I have to agree with Tim and
> his opinion of the Boot language. Dick Jenks told me that
> boot was to be the first step in implementing the entire
> system in the Spad language. Once the syntax looked like
> Spad, the next step would be to convert it to real Spad.
> Of course that never happened, so the main purpose behind
> Boot is gone.

If Boot is really dispensable, then I think Boot code should be raised to the
level of Spad, not lowered to the level of Lisp. I just cannot believe that
rewriting Boot code in Lisp would be more readable than Boot code in Spad. The
only exceptions are frequently inlined code should be optimized and there Lisp
may be a good choice.

I like to point out that Aldor has at least one intermediate language FOAM.
Probably the Aldor compiler compiles source to FOAM, which is then compiled to C
(correction requested since this is only a wild guess). Can Boot be modified to
be equivalent to FOAM in its role?
 
> In my opinion Boot can be a convenient language to code in,
> but it has two main problems. The use of indentation for
> grouping is just a bad idea. It works well for one-page
> programs, but with anything with reasonable complexity, it
> falls down.

I like the indentation (no pile) syntax and I agree that it took me a while to
get used to that and avoid the missing space errors. But I think whether you use
pile or no pile, it is only a matter of habit. As Bill Page pointed out, missing
a semicolon in other languages (Maple, for example) is equally devastating. In
fact I like the indentation so much that when I program in Pascal and other
languages, I put all the "parenthesize annoyances to the end of the line so as
not to interfere with my visual cues of the hierarchy -- for example:
     if not fileopen then                       begin
        writeln(outfilename,'  not found!');
        writeln('hit any key to continue');
        repeat until keypressed                 end;
which also maximizes the number of lines that can fit in one page. If I were
forced to use no pile, I would place the braces similarly to the begin/end pair.
I also agree with Bill that one should write a routine that spans more than one
page only if it is absolutely impossible to break up the code into smaller
chunks. Bill already noted that "A lot of the Spad code in the Axiom library is
less than one page per package domain or
category."


> the other "oh by the way" is that while you can seem to
> read boot code without knowing lisp (a sort-of pseudo
> code syntax) you can't WRITE boot code without knowing
> lisp.

Bill page responded:

> Why is that? Surely it is possible to write Spad code without
> knowing Lisp? That was the point of Spad, wasn't it? (Otherwise
> the mathematicians could have simply written in Lisp and this
> whole issue would be moot, right?) If Boot is a simplified
> form of Spad then why is knowing Lisp so relevant? And if by
> "knowing lisp" you mean knowing the concepts on which lisp is
> based, then I would be surprised to find many undergraduate
> computer science students who did not have at least one
> programming course which included at least an introduction
> to lisp. (Maybe my experience here is too out of date?)

I do not want to argue against Lisp (some would say Scheme is better). But I
agree with your implications from my own experience that writing algebra code in
Axiom requires absolutely no knowledge (ok, some reading knowledge, but only
because the compiler occasionally includes Lisp in error messages -- I don't
think anyone would disagree that the compiler needs to improve its diagnostic
messages) of Lisp or Boot. Of course, knowing these helps if the system
(compiler or interpreter) ever is in error; but that is for system people to
know. The algebra developers would simply report the bugs and meanwhile find a
way around them -- that was what I did most of the time because the system
developers in those days were busy with experimenting with many new ideas (in
particular Aldor or A#) and they said they would not fix those bugs or add some
natural features. But those days are over, I hope, with open source. If system
developers spend too much time on rewriting as a means to fix bugs, I believe
the bugs would not be fixed for a long time, and the new code will break old
code. Developing an algebra package is no trivial matter and it would be quite a
waste of resources to have to upgrade a package because of backward
incompatibility.  One should learn from the longevities of Fortran and Cobol.

> ---------
> 
> Now about this language war ...
> [snipped] 
> So, *if* I was thinking of such a compromise then there are
> some "terms of surrender" that might make this easier:
> 
> 1) Before any more "lispifying" goes on, *first* move Axiom
>    over to ANSI lisp. This will make it possible to write lisp
>    that conforms to modern practices and as high level as
>    possible. It will also make Axiom less dependent on a
>    particular implementation of Lisp and more easily portable.
> 
> 2) No more just cutting and pasting the generated lisp output
>    of Boot into the literate document! This seems pointless
>    and a net loss in intelligibility even if it is accompanied
>    by a verbal explanation of the code. The original Boot code
>    is, as Tim admitted above, significantly more readable as
>    "pseudo-code" then what the compiler outputs.
> 
> 3) Where possible and practical, replace Boot code with Spad
>    code as in Dick Jenks original vision instead of Lisp. Surely
>    you are not going to argue that Spad is a "dead language"
>    written by fewer than 10 surviving programmers? And even if
>    that happens to be true ;), that is one thing that we are
>    trying hard to change, right?

These terms should not be negotiable! Actually, I would put these as requests to
whoever does the work) I would add at least 

4) The error messages outputted by the compiler should NOT refer to any Lisp.
Nothing like the current $m, $x, $f stuff. All errors should refer as close to
the offending source code as possible. (I believe such output was due to
"laziness" or viewing more intelligible messages as an insult to the developers.
But that might be why Axiom failed to capture market shares in the 1990's.)

5) The )trace function should likewise return high level state of the algebra
variables, not lisp code or representation of them.

6) The interpreted language should adhere to the compiler language to facilitate
prototyping (in the interpreter) and production code (using the compiler) and
any deviations (due to the smartness of the interpreter) should be logged to
help the transfer (maybe it is even possible to translate a working interpreted
set of routines automatically into compilable code?).

William