RE: [Axiom-developer] Terms of Surrender (was: BAD tim)

[Bill Page]

Tim,

I feel rallied by William Sit's support, so here comes another
defensive action.

On November 6, 2005 1:55 AM you wrote:

> 
> > First about Boot. My (admittedly poor) understanding of 
> > Boot is that it is an intermedate language between Lisp
> > and Spad (and there are other intermediate languages Shoe,
> > Meta, etc as well). I agree that to build a truely high
> > level language like Spad/Axiom/Aldor, intermediate
> > languages are necessities and not evils. It's much like
> > what a mathematician does all the time (and most original
> > developers are mathematicians, in addition to being 
> > computer scientists): if the
> .....[snip].....
> 
> sorry, but you are reasoning by wrong analogy.
>

William's analogy seems perfectly reasonable to me. That
is exactly the kind of "mindset" and approach that seems
consistent with Axiom's origins. The point is building a
set of concepts, often embedded in a new language (Yes,
mathematicians do invent new language all the time!) which
make expressing the ideas and proofs that you really want
to express, that mush easier. Of course there is an overhead
for learning the new language, but in mathematics that is
often part of the "fun" and when it's done you realize that
you have made some important intellectual gain in power.

The situation of Boot and Lisp is similar because the
abstractions and concepts implicit in Spad would be quite
obscure if one attempted to express them directly in Lisp.
Boot *does* provide an intermediate ground between the
concrete implementation in Lisp and the abstraction of
Spad - just as Dick Jenks apparently intended.

> boot is not an intermediate step between lisp and spad.
> boot is a syntactic sugar coating on lisp that eliminates
> parentheses in favor of "piling" (indenting) code.

Tim, I am amazed that you would make this claim. It is quite
demonstratably incorrect. The use of parenthesis or not in Boot
is *completely* besides the point. Adding braces (as was done
in Aldor) is a trivial syntactic difference and *that* is what
I might agree could be characterized as simple sugar coating.
But the difference between boot and lisp, and the relationship
between boot and Spad is obvious. Even Scot Morrison seems to
hold this view. Just consider the example cited by Clifford Yap
many emails ago:

Boot language:

intloop () ==
    mode := $intRestart
    while mode = $intRestart repeat
      resetStackLimits()
      mode := CATCH($intTopLevel,
            SpadInterpretStream(1, ["TIM", "DALY", "?"], true))

Lisp:

(defun |intloop| ()
 (prog (mode)
  (declare (special |$intTopLevel| |$intRestart|))
   (return
    (progn
     (setq mode |$intRestart|)
     ((lambda ()
       (loop
        (cond
         ((not (equal mode |$intRestart|))
           (return nil))
         (t
          (progn
            (|resetStackLimits|)
            (setq mode
              (catch |$intTopLevel|
               (|SpadInterpretStream| 1
                 (list 'tim 'daly '?) t)))))))))))))

--------

Who could claim that the Boot code has a direct (sugar coating)
relationship to Lisp? Who would miss the similarity between
Boot and Spad?

> it adds no new functionality

Since when does any high level language "add functionality"?
You should not expect it too. Fortran does not add functionality
to Assembler language. If anything it specifically removes
functionality, trading-off extreme flexibility for a more
congenial model of computation. No one could argue that the
development of high level programming languages was some kind
of mistake.

> and does not help you reason about programming any more than,
> say, python or perl.

Here I also completely disagree with you. I *do* think that
Boot, Python and Perl do help you to reason about programming.
I don't mean in the "nuts and bolts" kind of way of say, a
'while'


> in fact it limits the things you can write so you can't use the full
> power of lisp.
> 
> boot does not function as a lemma-like level of reasoning.
> 
> > Second, I think once the bootstrap step is done, it is ok 
> to write a compiler
> > for the language in the language itself. So writing a 
> compiler for C in C, a
> > precompiler for C++ in C, one for Boot in Boot, or for 
> Axiom in Axiom is
> > perfectly reasonable. The purist may want to boil everything down to
> > hand-crafted assembly (dare I say machine?) language, 
> 
> .....[snip].....
> 
> swing and a miss, as they say in baseball. 
> 
> the interpreter needs to be:
>    simple to build, 
>    simple to maintain, 
>    simple to extend, 
>    simple to port.
> 
> writing boot in boot forces the system build to be more complex. 
> 
> it forces the maintenance to be harder since you need to understand 
> whether the problem is in the boot compiler or the interpreter. 
> 
> it forces the user to understand the interaction of lisp vs boot 
> when extending the system since certain facilities have to be in 
> lisp because boot is not yet available at that point in the build 
> process (e.g. setvars)
> 
> it makes the system harder to port because we have to rewrite the
> bootstrap lisp code from old common lisp to ansi common lisp 
> to port it.
> this makes the boot-level code out of date.
> 
> writing boot in boot is clever but it makes building, maintaining, 
> extending, and porting axiom harder than it has to be.
> 
> yes, this can be fixed but fixing it means that you have to
>   (1) rewrite the boot compiler changes in boot
>   (2) recompile the boot compiler to clisp
>   (3) clip out the changes from the clisp file
>   (4) insert the changes into the bootstrap sections of the 
> boot compiler
>   (5) rebuild the system
> 
> writing boot in boot is clever.
> we don't need clever code.
> 
> boot is NOT a higher level language than lisp. spad is.
> boot is NOT spad and is unrelated to spad.
> boot is ONLY seen by interpreter/compiler developers.
> boot is used by 1 developer (me) until the documentation completes.
> 
> 
> 
> > > the other "oh by the way" is that while you can seem to
> > > read boot code without knowing lisp (a sort-of pseudo
> > > code syntax) you can't WRITE boot code without knowing
> > > lisp.
> > 
> > Bill page responded:
> > 
> > > Why is that? Surely it is possible to write Spad code without
> > > knowing Lisp? 
> 
> because boot IS a limited version of lisp. for instance, in the
> history mechanism in bookvol5.pamphlet you'll see that boot uses RPLAC
> functions to create a circular list. to write this you need to
> understand that (a) you're using list structures, (b) how they are
> represented (as cons cells) and (c) how rplaca/rplacd modifies
> them. all of that requires an understanding of lisp, not boot.
> 
> > > 
> > > Now about this language war ...
> > > [snipped] 
> > > So, *if* I was thinking of such a compromise then there are
> > > some "terms of surrender" that might make this easier:
> > > 
> > > 1) Before any more "lispifying" goes on, *first* move Axiom
> > >    over to ANSI lisp. This will make it possible to write lisp
> > >    that conforms to modern practices and as high level as
> > >    possible. It will also make Axiom less dependent on a
> > >    particular implementation of Lisp and more easily portable.
> 
> so you're suggesting that we modify the boot compiler to generate ansi
> lisp code? yet during the "lispifying" rewrite i AM rewriting to ansi
> lisp.
> 
> > > 
> > > 2) No more just cutting and pasting the generated lisp output
> > >    of Boot into the literate document! This seems pointless
> > >    and a net loss in intelligibility even if it is accompanied
> > >    by a verbal explanation of the code. The original Boot code
> > >    is, as Tim admitted above, significantly more readable as
> > >    "pseudo-code" then what the compiler outputs.
> 
> the cut and paste has to happen. it's the only way to make sure the
> code does what it originally did. successive passes of the code have
> been complete rewrites. you'll see this at the next code release.
> there are a LOT of subtle issues that require careful thought and
> having working lisp code, even clipped code, is a necessary guide.
> 
> believe me, i find the boot compiler output code even more painful
> than anyone else. but it is important to move the code from a
> correct state to a correct state during the rewrite and i'm not
> man enough to do it all in one go.
> 
> (btw, as a side-effect of the rewrite i've already found and
> corrected two bugs).
> 
> if you tried to maintain the system as it is now you'll find that
> the boot compiler output is all over the source tree. try this:
> 
>   cd axiom/src
>   fgrep -r DEFUN *
> 
> the boot compiler generates an uppercase DEFUN. look at how
> pervasive the compiled lisp code has to be to actually get the
> system to bootstrap. in every file where the DEFUN occurs
> the BOOT code is just PSUEDO-CODE.
> 
> in particular, 
> 
>   cd axiom/src/boot
>   fgrep DEFUN *
> 
> and you'll see how many of the boot compiler files aren't REALLY
> written in boot.
> 
> believe me, it took a long time with a lot of effort to get the
> system to build without a running axiom system.
> 
>  and it ain't pretty. but it should be.
> 
> > > 
> > > 3) Where possible and practical, replace Boot code with Spad
> > >    code as in Dick Jenks original vision instead of Lisp. Surely
> > >    you are not going to argue that Spad is a "dead language"
> > >    written by fewer than 10 surviving programmers? And even if
> > >    that happens to be true ;), that is one thing that we are
> > >    trying hard to change, right?
> 
> wrong level. boot is NOT spad. the languages are NOT related except
> by authors. they have different design goals. 
> 
> > 
> > These terms should not be negotiable! Actually, I would put 
> these as 
> > requests to whoever does the work) I would add at least 
> 
> ummm, i'm doing the work :-)
> 
> and i REALLY don't want to rewrite the boot compiler to make it do
> things like ansi common lisp generation. especially if, after having
> achieved the 3 *terms* i'd would then have agreement that i could
> elide boot from the system. what would be the point?
> 
> > 
> > 4) The error messages outputted by the compiler should NOT 
> refer to any Lisp.
> > Nothing like the current $m, $x, $f stuff. All errors 
> should refer as close to
> > the offending source code as possible. (I believe such 
> output was due to
> > "laziness" or viewing more intelligible messages as an 
> insult to the developers.
> > But that might be why Axiom failed to capture market shares 
> in the 1990's.)
> 
> the output uses the variables like $m because that's all the 
> developers
> need to know to debug. $m is the target mode information that the
> compiler is trying to achieve. 
> 
> i agree that the compiler needs to be rewritten to do things like
> generate better error messages. 
>   to do THAT level of rewrite requires a deep understanding 
> of the compiler. 
>     to achieve THAT level of understanding we need to deeply 
> document how 
>     the compiler works. 
>       to do THAT level of documentation we need to review 
> each function 
>       in detail. 
>         to do THAT detailed level of documentation is my 
> current goal. 
>         as a side-effect of achieving that level of understanding the
>         code will be in lisp (which is a language i deeply 
> understand.)
> 
> > 
> > 5) The )trace function should likewise return high level 
> state of the algebra
> > variables, not lisp code or representation of them.
> 
> admirable goal. BIG changes to the interpreter and compiler data
> structures (since the code is COMPILED into machine language) to
> maintain state information that trace can use.
> 
> and it needs to be done without impacting efficiency, accumulating
> large amounts of possibly unneeded state, etc.
> 
> this is like asking C to give you only algorithmicly related error
> messages. 
> 
> 
> > 6) The interpreted language should adhere to the compiler 
> language to facilitate
> > prototyping (in the interpreter) and production code (using 
> the compiler) and
> > any deviations (due to the smartness of the interpreter) 
> should be logged to
> > help the transfer (maybe it is even possible to translate a 
> working interpreted
> > set of routines automatically into compilable code?).
> 
> we need to "reify" (name) the interpreter type-translation, guessing
> and manipulation mechanisms. we need to separate them from the
> interpreter itself so we can understand, add or modify them. 
> 
> ( to do THAT level of rewrite requires a deep understanding 
> of the interpreter
>   ( to achieve THAT level of understanding we need to deeply 
> document how 
>     the interpreter works. 
>      ( to do THAT level of documentation we need to review 
> each function 
>        in detail. 
>        ( to do THAT detailed level of documentation is my 
> current goal. 
>          as a side-effect of achieving that level of understanding the
>          code will be in lisp (which is a language i deeply 
> understand.)))))
> 
> 
> read the book when it comes out.
> the story turns out ok in the end.
> 
> BAD Tim
> 
> 
>