[Guile-commits] GNU Guile branch, master, updated. v2.1.0-528-g67915ab

[Andy Wingo]

This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "GNU Guile".

http://git.savannah.gnu.org/cgit/guile.git/commit/?id=67915ab079c806251232afe115c22444ee31d8af

The branch, master has been updated
       via  67915ab079c806251232afe115c22444ee31d8af (commit)
      from  691697de0937c52488952c1d79d3ba4416ccbb0f (commit)

Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.

- Log -----------------------------------------------------------------
commit 67915ab079c806251232afe115c22444ee31d8af
Author: Andy Wingo <address@hidden>
Date:   Wed Dec 4 20:46:02 2013 +0100

    Doc updates to macroexpansion, compiled procs, and compiler.texi
    
    * doc/ref/api-macros.texi (Macro Expansion): New section.
    * doc/ref/api-procedures.texi (Compiled Procedures): Beginnings of a
      revision.  Not finished.
    * doc/ref/compiler.texi (Compiling to the Virtual Machine): Beginnings
      of a revision.  CPS and bytecode are not done yet.

-----------------------------------------------------------------------

Summary of changes:
 doc/ref/api-macros.texi     |   67 ++++++
 doc/ref/api-procedures.texi |   48 ++---
 doc/ref/compiler.texi       |  514 +++++++++----------------------------------
 3 files changed, 191 insertions(+), 438 deletions(-)

diff --git a/doc/ref/api-macros.texi b/doc/ref/api-macros.texi
index 82a2c07..6eb41c6 100644
--- a/doc/ref/api-macros.texi
+++ b/doc/ref/api-macros.texi
@@ -43,6 +43,7 @@ languages}, or EDSLs.}.
 * Identifier Macros::           Identifier macros.
 * Syntax Parameters::           Syntax Parameters.
 * Eval When::                   Affecting the expand-time environment.
+* Macro Expansion::             Procedurally expanding macros.
 * Internal Macros::             Macros as first-class values.
 @end menu
 
@@ -1122,6 +1123,72 @@ Evaluate @var{exp...} under the given @var{conditions}. 
Valid conditions include
 Other uses of @code{eval-when} may void your warranty or poison your cat.
 @end deffn
 
address@hidden Macro Expansion
address@hidden Macro Expansion
+
+Usually, macros are expanded on behalf of the user as needed.  Macro
+expansion is an integral part of @code{eval} and @code{compile}.  Users
+can also expand macros at the REPL prompt via the @code{expand} REPL
+command; @xref{Compile Commands}.
+
+Macros can also be expanded programmatically, via @code{macroexpand},
+but the details get a bit hairy for two reasons.
+
+The first complication is that the result of macro-expansion isn't
+Scheme: it's Tree-IL, Guile's high-level intermediate language.
address@hidden  As ``hygienic macros'' can produce identifiers that are
+distinct but have the same name, the output format needs to be able to
+represent distinctions between variable identities and names.  Again,
+see @xref{Tree-IL} for all the details.  The easiest thing is to just
+run @code{tree-il->scheme} on the result of macro-expansion:
+
address@hidden
+(macroexpand '(+ 1 2))
address@hidden
+#<tree-il (call (toplevel +) (const 1) (const 2))>
+
+(use-modules (language tree-il))
+(tree-il->scheme (macroexpand '(+ 1 2)))
address@hidden
+(+ 1 2)
address@hidden lisp
+
+The second complication involves @code{eval-when}.  As an example, what
+would it mean to macro-expand the definition of a macro?
+
address@hidden
+(macroexpand '(define-syntax qux (identifier-syntax 'bar)))
address@hidden
+?
address@hidden lisp
+
+The answer is that it depends who is macro-expanding, and why.  Do you
+define the macro in the current environment?  Residualize a macro
+definition?  Both?  Neither?  The default is to expand in ``eval'' mode,
+which means an @code{eval-when} clauses will only proceed when
address@hidden (or @code{expand}) is in its condition set.  Top-level
+macros will be @code{eval}'d in the top-level environment.
+
+In this way @code{(macroexpand @var{foo})} is equivalent to
address@hidden(macroexpand @var{foo} 'e '(eval))}.  The second argument is the
+mode (@code{'e} for ``eval'') and the second is the
+eval-syntax-expanders-when parameter (only @code{eval} in this default
+setting).
+
+But if you are compiling the macro definition, probably you want to
+reify the macro definition itself.  In that case you pass @code{'c} as
+the second argument to @code{macroexpand}.  But probably you want the
+macro definition to be present at compile time as well, so you pass
address@hidden'(compile load eval)} as the @var{esew} parameter.  In fact
address@hidden(compile @var{foo} #:to 'tree-il)} is entirely equivalent to
address@hidden(macroexpand @var{foo} 'c '(compile load eval))}; @xref{The Scheme
+Compiler}.
+
+It's a terrible interface; we know.  The macroexpander is somewhat
+tricksy regarding modes, so unless you are building a macro-expanding
+tool, we suggest to avoid invoking it directly.
+
+
 @node Internal Macros
 @subsection Internal Macros
 
diff --git a/doc/ref/api-procedures.texi b/doc/ref/api-procedures.texi
index 7fedadf..02bf682 100644
--- a/doc/ref/api-procedures.texi
+++ b/doc/ref/api-procedures.texi
@@ -161,48 +161,30 @@ Returns @code{#t} if @var{obj} is a compiled procedure, 
or @code{#f}
 otherwise.
 @end deffn
 
address@hidden {Scheme Procedure} program-objcode program
address@hidden {C Function} scm_program_objcode (program)
-Returns the object code associated with this program. @xref{Bytecode
-and Objcode}, for more information.
address@hidden {Scheme Procedure} program-code program
address@hidden {C Function} scm_program_code (program)
+Returns the address of the program's entry, as an integer.  This address
+is mostly useful to procedures in @code{(system vm debug)}.
 @end deffn
 
address@hidden {Scheme Procedure} program-objects program
address@hidden {C Function} scm_program_objects (program)
-Returns the ``object table'' associated with this program, as a
-vector. @xref{VM Programs}, for more information.
address@hidden {Scheme Procedure} program-num-free-variable program
address@hidden {C Function} scm_program_num_free_variables (program)
+Return the number of free variables captured by this program.
 @end deffn
 
address@hidden {Scheme Procedure} program-module program
address@hidden {C Function} scm_program_module (program)
-Returns the module that was current when this program was created. Can
-return @code{#f} if the compiler could determine that this information
-was unnecessary.
address@hidden deffn
-
address@hidden {Scheme Procedure} program-free-variables program
address@hidden {C Function} scm_program_free_variables (program)
-Returns the set of free variables that this program captures in its
-closure, as a vector. If a closure is code with data, you can get the
-code from @code{program-objcode}, and the data via
address@hidden
-
-Some of the values captured are actually in variable ``boxes''.
address@hidden and the VM}, for more information.
address@hidden {Scheme Procedure} program-free-variable-ref program n
address@hidden {C Function} scm_program_free_variable-ref (program, n)
address@hidden {Scheme Procedure} program-free-variable-set! program n val
address@hidden {C Function} scm_program_free_variable_set_x (program, n, val)
+Accessors for a program's free variables.  Some of the values captured
+are actually in variable ``boxes''.  @xref{Variables and the VM}, for
+more information.
 
 Users must not modify the returned value unless they think they're
 really clever.
 @end deffn
 
address@hidden {Scheme Procedure} program-meta program
address@hidden {C Function} scm_program_meta (program)
-Return the metadata thunk of @var{program}, or @code{#f} if it has no
-metadata.
-
-When called, a metadata thunk returns a list of the following form:
address@hidden(@var{bindings} @var{sources} @var{arities} . @var{properties})}. 
The format
-of each of these elements is discussed below.
address@hidden deffn
address@hidden FIXME
 
 @deffn {Scheme Procedure} program-bindings program
 @deffnx {Scheme Procedure} make-binding name boxed? index start end
diff --git a/doc/ref/compiler.texi b/doc/ref/compiler.texi
index 553c270..235e6c1 100644
--- a/doc/ref/compiler.texi
+++ b/doc/ref/compiler.texi
@@ -7,14 +7,11 @@
 @node Compiling to the Virtual Machine
 @section Compiling to the Virtual Machine
 
-Compilers have a mystique about them that is attractive and
-off-putting at the same time. They are attractive because they are
-magical -- they transform inert text into live results, like throwing
-the switch on Frankenstein's monster. However, this magic is perceived
-by many to be impenetrable.
-
-This section aims to pay attention to the small man behind the
-curtain.
+Compilers!  The word itself inspires excitement and awe, even among
+experienced practitioners.  But a compiler is just a program: an
+eminently hackable thing.  This section aims to to describe Guile's
+compiler in such a way that interested Scheme hackers can feel
+comfortable reading and extending it.
 
 @xref{Read/Load/Eval/Compile}, if you're lost and you just wanted to
 know how to compile your @code{.scm} file.
@@ -23,9 +20,8 @@ know how to compile your @code{.scm} file.
 * Compiler Tower::                   
 * The Scheme Compiler::                   
 * Tree-IL::                 
-* GLIL::                
-* Assembly::                   
-* Bytecode and Objcode::                   
+* Continuation-Passing Style::                 
+* Bytecode::                
 * Writing New High-Level Languages::
 * Extending the Compiler::
 @end menu
@@ -34,15 +30,14 @@ know how to compile your @code{.scm} file.
 @subsection Compiler Tower
 
 Guile's compiler is quite simple, actually -- its @emph{compilers}, to
-put it more accurately. Guile defines a tower of languages, starting
-at Scheme and progressively simplifying down to languages that
-resemble the VM instruction set (@pxref{Instruction Set}).
+put it more accurately.  Guile defines a tower of languages, starting at
+Scheme and progressively simplifying down to languages that resemble the
+VM instruction set (@pxref{Instruction Set}).
 
 Each language knows how to compile to the next, so each step is simple
-and understandable. Furthermore, this set of languages is not
-hardcoded into Guile, so it is possible for the user to add new
-high-level languages, new passes, or even different compilation
-targets.
+and understandable.  Furthermore, this set of languages is not hardcoded
+into Guile, so it is possible for the user to add new high-level
+languages, new passes, or even different compilation targets.
 
 Languages are registered in the module, @code{(system base language)}:
 
@@ -60,10 +55,10 @@ They are registered with the @code{define-language} form.
                        [#:make-default-environment=make-fresh-user-module]
 Define a language.
 
-This syntax defines a @code{#<language>} object, bound to @var{name}
-in the current environment. In addition, the language will be added to
-the global language set. For example, this is the language definition
-for Scheme:
+This syntax defines a @code{<language>} object, bound to @var{name} in
+the current environment.  In addition, the language will be added to the
+global language set.  For example, this is the language definition for
+Scheme:
 
 @example
 (define-language scheme
@@ -78,7 +73,7 @@ for Scheme:
 @end deffn
 
 The interesting thing about having languages defined this way is that
-they present a uniform interface to the read-eval-print loop. This
+they present a uniform interface to the read-eval-print loop.  This
 allows the user to change the current language of the REPL:
 
 @example
@@ -116,8 +111,8 @@ fast.
 
 There is a notion of a ``current language'', which is maintained in the
 @code{current-language} parameter, defined in the core @code{(guile)}
-module. This language is normally Scheme, and may be rebound by the
-user. The run-time compilation interfaces
+module.  This language is normally Scheme, and may be rebound by the
+user.  The run-time compilation interfaces
 (@pxref{Read/Load/Eval/Compile}) also allow you to choose other source
 and target languages.
 
@@ -126,30 +121,31 @@ The normal tower of languages when compiling Scheme goes 
like this:
 @itemize
 @item Scheme
 @item Tree Intermediate Language (Tree-IL)
address@hidden Guile Lowlevel Intermediate Language (GLIL)
address@hidden Assembly
address@hidden Continuation-Passing Style (CPS)
 @item Bytecode
address@hidden Objcode
 @end itemize
 
-Object code may be serialized to disk directly, though it has a cookie
-and version prepended to the front. But when compiling Scheme at run
-time, you want a Scheme value: for example, a compiled procedure. For
-this reason, so as not to break the abstraction, Guile defines a fake
-language at the bottom of the tower:
+As discussed before (@pxref{Object File Format}), bytecode is in ELF
+format, ready to be serialized to disk.  But when compiling Scheme at
+run time, you want a Scheme value: for example, a compiled procedure.
+For this reason, so as not to break the abstraction, Guile defines a
+fake language at the bottom of the tower:
 
 @itemize
 @item Value
 @end itemize
 
-Compiling to @code{value} loads the object code into a procedure, and
-wakes the sleeping giant.
+Compiling to @code{value} loads the bytecode into a procedure, and wakes
+the sleeping giant.
 
 Perhaps this strangeness can be explained by example:
 @code{compile-file} defaults to compiling to object code, because it
 produces object code that has to live in the barren world outside the
-Guile runtime; but @code{compile} defaults to compiling to
address@hidden, as its product re-enters the Guile world.
+Guile runtime; but @code{compile} defaults to compiling to @code{value},
+as its product re-enters the Guile world.
+
address@hidden FIXME: This doesn't work anymore :(  Should we add some kind of
address@hidden special GC pass, or disclaim this kind of code, or what?
 
 Indeed, the process of compilation can circulate through these
 different worlds indefinitely, as shown by the following quine:
@@ -162,20 +158,20 @@ different worlds indefinitely, as shown by the following 
quine:
 @subsection The Scheme Compiler
 
 The job of the Scheme compiler is to expand all macros and all of Scheme
-to its most primitive expressions. The definition of ``primitive'' is
-given by the inventory of constructs provided by Tree-IL, the target
-language of the Scheme compiler: procedure calls, conditionals, lexical
-references, etc. This is described more fully in the next section.
+to its most primitive expressions.  The definition of ``primitive
+expression'' is given by the inventory of constructs provided by
+Tree-IL, the target language of the Scheme compiler: procedure calls,
+conditionals, lexical references, and so on.  This is described more
+fully in the next section.
 
 The tricky and amusing thing about the Scheme-to-Tree-IL compiler is
-that it is completely implemented by the macro expander. Since the
+that it is completely implemented by the macro expander.  Since the
 macro expander has to run over all of the source code already in order
 to expand macros, it might as well do the analysis at the same time,
 producing Tree-IL expressions directly.
 
-Because this compiler is actually the macro expander, it is
-extensible. Any macro which the user writes becomes part of the
-compiler.
+Because this compiler is actually the macro expander, it is extensible.
+Any macro which the user writes becomes part of the compiler.
 
 The Scheme-to-Tree-IL expander may be invoked using the generic
 @code{compile} procedure:
@@ -183,38 +179,16 @@ The Scheme-to-Tree-IL expander may be invoked using the 
generic
 @lisp
 (compile '(+ 1 2) #:from 'scheme #:to 'tree-il)
 @result{}
- #<<call> src: #f
-          proc: #<<toplevel-ref> src: #f name: +>
-          args: (#<<const> src: #f exp: 1>
-                 #<<const> src: #f exp: 2>)>
address@hidden lisp
-
-Or, since Tree-IL is so close to Scheme, it is often useful to expand
-Scheme to Tree-IL, then translate back to Scheme. For that reason the
-expander provides two interfaces. The former is equivalent to calling
address@hidden(macroexpand '(+ 1 2) 'c)}, where the @code{'c} is for
-``compile''. With @code{'e} (the default), the result is translated
-back to Scheme:
-
address@hidden
-(macroexpand '(+ 1 2))
address@hidden (+ 1 2)
-(macroexpand '(let ((x 10)) (* x x)))
address@hidden (let ((x84 10)) (* x84 x84))
+#<tree-il (call (toplevel +) (const 1) (const 2))>
 @end lisp
 
-The second example shows that as part of its job, the macro expander
-renames lexically-bound variables. The original names are preserved
-when compiling to Tree-IL, but can't be represented in Scheme: a
-lexical binding only has one name. It is for this reason that the
address@hidden output of the expander is @emph{not} Scheme. There's too
-much information we would lose if we translated to Scheme directly:
-lexical variable names, source locations, and module hygiene.
-
-Note however that @code{macroexpand} does not have the same signature
-as @code{compile-tree-il}. @code{compile-tree-il} is a small wrapper
-around @code{macroexpand}, to make it conform to the general form of
-compiler procedures in Guile's language tower.
address@hidden(compile @var{foo} #:from 'scheme #:to 'tree-il)} is entirely
+equivalent to calling the macro expander as @code{(macroexpand @var{foo}
+'c '(compile load eval))}.  @xref{Macro Expansion}.
address@hidden, the procedure dispatched by @code{compile} to
address@hidden'tree-il}, is a small wrapper around @code{macroexpand}, to make
+it conform to the general form of compiler procedures in Guile's
+language tower.
 
 Compiler procedures take three arguments: an expression, an
 environment, and a keyword list of options. They return three values:
@@ -309,7 +283,7 @@ Users may program with this format directly at the REPL:
 @example
 scheme@@(guile-user)> ,language tree-il
 Happy hacking with Tree Intermediate Language!  To switch back, type `,L 
scheme'.
-tree-il@@(guile-user)> (apply (primitive +) (const 32) (const 10))
+tree-il@@(guile-user)> (call (primitive +) (const 32) (const 10))
 @result{} 42
 @end example
 
@@ -325,7 +299,7 @@ take care of the rest.
 
 @deftp {Scheme Variable} <void> src
 @deftpx {External Representation} (void)
-An empty expression. In practice, equivalent to Scheme's @code{(if #f
+An empty expression.  In practice, equivalent to Scheme's @code{(if #f
 #f)}.
 @end deftp
 @deftp {Scheme Variable} <const> src exp
@@ -334,20 +308,20 @@ A constant.
 @end deftp
 @deftp {Scheme Variable} <primitive-ref> src name
 @deftpx {External Representation} (primitive @var{name})
-A reference to a ``primitive''. A primitive is a procedure that, when
-compiled, may be open-coded. For example, @code{cons} is usually
+A reference to a ``primitive''.  A primitive is a procedure that, when
+compiled, may be open-coded.  For example, @code{cons} is usually
 recognized as a primitive, so that it compiles down to a single
 instruction.
 
 Compilation of Tree-IL usually begins with a pass that resolves some
 @code{<module-ref>} and @code{<toplevel-ref>} expressions to
address@hidden<primitive-ref>} expressions. The actual compilation pass has
address@hidden<primitive-ref>} expressions.  The actual compilation pass has
 special cases for calls to certain primitives, like @code{apply} or
 @code{cons}.
 @end deftp
 @deftp {Scheme Variable} <lexical-ref> src name gensym
 @deftpx {External Representation} (lexical @var{name} @var{gensym})
-A reference to a lexically-bound variable. The @var{name} is the
+A reference to a lexically-bound variable.  The @var{name} is the
 original name of the variable in the source program. @var{gensym} is a
 unique identifier for this variable.
 @end deftp
@@ -400,15 +374,17 @@ analyze than @code{<call>}.
 As part of the compilation process, instances of @code{(call (primitive
 @var{name}) . @var{args})} are transformed into primcalls.
 @end deftp
address@hidden {Scheme Variable} <sequence> src exps
address@hidden {External Representation} (begin . @var{exps})
-Like Scheme's @code{begin}.
address@hidden {Scheme Variable} <seq> src head tail
address@hidden {External Representation} (seq @var{head} @var{tail})
+A sequence.  The semantics is that @var{head} is evaluated first, and
+any resulting values are ignored.  Then @var{tail} is evaluated, in tail
+position.
 @end deftp
 @deftp {Scheme Variable} <lambda> src meta body
 @deftpx {External Representation} (lambda @var{meta} @var{body})
-A closure. @var{meta} is an association list of properties for the
-procedure. @var{body} is a single Tree-IL expression of type
address@hidden<lambda-case>}. As the @code{<lambda-case>} clause can chain to
+A closure.  @var{meta} is an association list of properties for the
+procedure.  @var{body} is a single Tree-IL expression of type
address@hidden<lambda-case>}.  As the @code{<lambda-case>} clause can chain to
 an alternate clause, this makes Tree-IL's @code{<lambda>} have the
 expressiveness of Scheme's @code{case-lambda}.
 @end deftp
@@ -417,7 +393,7 @@ expressiveness of Scheme's @code{case-lambda}.
   (lambda-case ((@var{req} @var{opt} @var{rest} @var{kw} @var{inits} 
@var{gensyms})@
                 @var{body})@
                address@hidden)
-One clause of a @code{case-lambda}. A @code{lambda} expression in
+One clause of a @code{case-lambda}.  A @code{lambda} expression in
 Scheme is treated as a @code{case-lambda} with one clause.
 
 @var{req} is a list of the procedure's required arguments, as symbols.
@@ -428,9 +404,9 @@ argument, or @code{#f}.
 @var{kw} is a list of the form, @code{(@var{allow-other-keys?}
 (@var{keyword} @var{name} @var{var}) ...)}, where @var{keyword} is the
 keyword corresponding to the argument named @var{name}, and whose
-corresponding gensym is @var{var}. @var{inits} are tree-il expressions
-corresponding to all of the optional and keyword arguments, evaluated
-to bind variables whose value is not supplied by the procedure caller.
+corresponding gensym is @var{var}.  @var{inits} are tree-il expressions
+corresponding to all of the optional and keyword arguments, evaluated to
+bind variables whose value is not supplied by the procedure caller.
 Each @var{init} expression is evaluated in the lexical context of
 previously bound variables, from left to right.
 
@@ -438,17 +414,17 @@ previously bound variables, from left to right.
 first all of the required arguments, then the optional arguments if
 any, then the rest argument if any, then all of the keyword arguments.
 
address@hidden is the body of the clause. If the procedure is called with
address@hidden is the body of the clause.  If the procedure is called with
 an appropriate number of arguments, @var{body} is evaluated in tail
-position. Otherwise, if there is an @var{alternate}, it should be a
+position.  Otherwise, if there is an @var{alternate}, it should be a
 @code{<lambda-case>} expression, representing the next clause to try.
 If there is no @var{alternate}, a wrong-number-of-arguments error is
 signaled.
 @end deftp
 @deftp {Scheme Variable} <let> src names gensyms vals exp
 @deftpx {External Representation} (let @var{names} @var{gensyms} @var{vals} 
@var{exp})
-Lexical binding, like Scheme's @code{let}. @var{names} are the
-original binding names, @var{gensyms} are gensyms corresponding to the
+Lexical binding, like Scheme's @code{let}.  @var{names} are the original
+binding names, @var{gensyms} are gensyms corresponding to the
 @var{names}, and @var{vals} are Tree-IL expressions for the values.
 @var{exp} is a single Tree-IL expression.
 @end deftp
@@ -458,23 +434,25 @@ original binding names, @var{gensyms} are gensyms 
corresponding to the
 A version of @code{<let>} that creates recursive bindings, like
 Scheme's @code{letrec}, or @code{letrec*} if @var{in-order?} is true.
 @end deftp
address@hidden {Scheme Variable} <prompt> tag body handler
address@hidden {External Representation} (prompt @var{tag} @var{body} 
@var{handler})
-A dynamic prompt. Instates a prompt named @var{tag}, an expression,
address@hidden {Scheme Variable} <prompt> escape-only? tag body handler
address@hidden {External Representation} (prompt @var{escape-only?} @var{tag} 
@var{body} @var{handler})
+A dynamic prompt.  Instates a prompt named @var{tag}, an expression,
 during the dynamic extent of the execution of @var{body}, also an
-expression. If an abort occurs to this prompt, control will be passed
-to @var{handler}, a @code{<lambda-case>} expression with no optional
-or keyword arguments, and no alternate. The first argument to the
address@hidden<lambda-case>} will be the captured continuation, and then all
-of the values passed to the abort. @xref{Prompts}, for more
-information.
+expression.  If an abort occurs to this prompt, control will be passed
+to @var{handler}, also an expression, which should be a procedure.  The
+first argument to the handler procedure will be the captured
+continuation, followed by all of the values passed to the abort.  If
address@hidden is true, the handler should be a @code{<lambda>} with
+a single @code{<lambda-case>} body expression with no optional or
+keyword arguments, and no alternate, and whose first argument is
+unreferenced.  @xref{Prompts}, for more information.
 @end deftp
 @deftp {Scheme Variable} <abort> tag args tail
 @deftpx {External Representation} (abort @var{tag} @var{args} @var{tail})
 An abort to the nearest prompt with the name @var{tag}, an expression.
 @var{args} should be a list of expressions to pass to the prompt's
 handler, and @var{tail} should be an expression that will evaluate to
-a list of additional arguments. An abort will save the partial
+a list of additional arguments.  An abort will save the partial
 continuation, which may later be reinstated, resulting in the
 @code{<abort>} expression evaluating to some number of values.
 @end deftp
@@ -482,15 +460,15 @@ continuation, which may later be reinstated, resulting in 
the
 There are two Tree-IL constructs that are not normally produced by
 higher-level compilers, but instead are generated during the
 source-to-source optimization and analysis passes that the Tree-IL
-compiler does. Users should not generate these expressions directly,
-unless they feel very clever, as the default analysis pass will
-generate them as necessary.
+compiler does.  Users should not generate these expressions directly,
+unless they feel very clever, as the default analysis pass will generate
+them as necessary.
 
 @deftp {Scheme Variable} <let-values> src names gensyms exp body
 @deftpx {External Representation} (let-values @var{names} @var{gensyms} 
@var{exp} @var{body})
 Like Scheme's @code{receive} -- binds the values returned by
 evaluating @code{exp} to the @code{lambda}-like bindings described by
address@hidden That is to say, @var{gensyms} may be an improper list.
address@hidden  That is to say, @var{gensyms} may be an improper list.
 
 @code{<let-values>} is an optimization of a @code{<call>} to the
 primitive, @code{call-with-values}.
@@ -503,284 +481,38 @@ Like @code{<letrec>}, but only for @var{vals} that are 
unset
 @code{fix} is an optimization of @code{letrec} (and @code{let}).
 @end deftp
 
-Tree-IL implements a compiler to GLIL that recursively traverses
-Tree-IL expressions, writing out GLIL expressions into a linear list.
-The compiler also keeps some state as to whether the current
-expression is in tail context, and whether its value will be used in
-future computations. This state allows the compiler not to emit code
-for constant expressions that will not be used (e.g.@: docstrings), and
-to perform tail calls when in tail position.
-
-Most optimization, such as it currently is, is performed on Tree-IL
-expressions as source-to-source transformations. There will be more
-optimizations added in the future.
-
-Interested readers are encouraged to read the implementation in
address@hidden(language tree-il compile-glil)} for more details.
-
address@hidden GLIL
address@hidden GLIL
-
-Guile Lowlevel Intermediate Language (GLIL) is a structured intermediate
-language whose expressions more closely approximate Guile's VM
-instruction set. Its expression types are defined in @code{(language
-glil)}.
-
address@hidden {Scheme Variable} <glil-program> meta . body
-A unit of code that at run-time will correspond to a compiled
-procedure. @var{meta} should be an alist of properties, as in
-Tree-IL's @code{<lambda>}. @var{body} is an ordered list of GLIL
-expressions.
address@hidden deftp
address@hidden {Scheme Variable} <glil-std-prelude> nreq nlocs else-label
-A prologue for a function with no optional, keyword, or rest
-arguments. @var{nreq} is the number of required arguments. @var{nlocs}
-the total number of local variables, including the arguments. If the
-procedure was not given exactly @var{nreq} arguments, control will
-jump to @var{else-label}, if given, or otherwise signal an error.
address@hidden deftp
address@hidden {Scheme Variable} <glil-opt-prelude> nreq nopt rest nlocs 
else-label
-A prologue for a function with optional or rest arguments. Like
address@hidden<glil-std-prelude>}, with the addition that @var{nopt} is the
-number of optional arguments (possibly zero) and @var{rest} is an
-index of a local variable at which to bind a rest argument, or
address@hidden if there is no rest argument.
address@hidden deftp
address@hidden {Scheme Variable} <glil-kw-prelude> nreq nopt rest kw 
allow-other-keys? nlocs else-label
-A prologue for a function with keyword arguments. Like
address@hidden<glil-opt-prelude>}, with the addition that @var{kw} is a list
-of keyword arguments, and @var{allow-other-keys?} is a flag indicating
-whether to allow unknown keys. @xref{Function Prologue Instructions,
address@hidden, for details on the format of @var{kw}.
address@hidden deftp
address@hidden {Scheme Variable} <glil-bind> . vars
-An advisory expression that notes a liveness extent for a set of
-variables. @var{vars} is a list of @code{(@var{name} @var{type}
address@hidden)}, where @var{type} should be either @code{argument},
address@hidden, or @code{external}.
-
address@hidden<glil-bind>} expressions end up being serialized as part of a
-program's metadata and do not form part of a program's code path.
address@hidden deftp
address@hidden {Scheme Variable} <glil-mv-bind> vars rest
-A multiple-value binding of the values on the stack to @var{vars}.  If
address@hidden is true, the last element of @var{vars} will be treated as a
-rest argument.
-
-In addition to pushing a binding annotation on the stack, like
address@hidden<glil-bind>}, an expression is emitted at compilation time to
-make sure that there are enough values available to bind. See the
-notes on @code{truncate-values} in @ref{Procedure Call and Return
-Instructions}, for more information.
address@hidden deftp
address@hidden {Scheme Variable} <glil-unbind>
-Closes the liveness extent of the most recently encountered
address@hidden<glil-bind>} or @code{<glil-mv-bind>} expression. As GLIL
-expressions are compiled, a parallel stack of live bindings is
-maintained; this expression pops off the top element from that stack.
-
-Bindings are written into the program's metadata so that debuggers and
-other tools can determine the set of live local variables at a given
-offset within a VM program.
address@hidden deftp
address@hidden {Scheme Variable} <glil-source> loc
-Records source information for the preceding expression. @var{loc}
-should be an association list of containing @code{line} @code{column},
-and @code{filename} keys, e.g.@: as returned by
address@hidden
address@hidden deftp
address@hidden {Scheme Variable} <glil-void>
-Pushes ``the unspecified value'' on the stack.
address@hidden deftp
address@hidden {Scheme Variable} <glil-const> obj
-Pushes a constant value onto the stack. @var{obj} must be a number,
-string, symbol, keyword, boolean, character, uniform array, the empty
-list, or a pair or vector of constants.
address@hidden deftp
address@hidden {Scheme Variable} <glil-lexical> local? boxed? op index
-Accesses a lexically bound variable. If the variable is not
address@hidden it is free. All variables may have @code{ref},
address@hidden, and @code{bound?} as their @var{op}. Boxed variables may
-also have the @var{op}s @code{box}, @code{empty-box}, and @code{fix},
-which correspond in semantics to the VM instructions @code{box},
address@hidden, and @code{fix-closure}. @xref{Stack Layout}, for
-more information.
address@hidden deftp
address@hidden {Scheme Variable} <glil-toplevel> op name
-Accesses a toplevel variable. @var{op} may be @code{ref}, @code{set},
-or @code{define}.
address@hidden deftp
address@hidden {Scheme Variable} <glil-module> op mod name public?
-Accesses a variable within a specific module. See Tree-IL's
address@hidden<module-ref>}, for more information.
address@hidden deftp
address@hidden {Scheme Variable} <glil-label> label
-Creates a new label. @var{label} can be any Scheme value, and should
-be unique.
address@hidden deftp
address@hidden {Scheme Variable} <glil-branch> inst label
-Branch to a label. @var{label} should be a @code{<ghil-label>}.
address@hidden is a branching instruction: @code{br-if}, @code{br}, etc.
address@hidden deftp
address@hidden {Scheme Variable} <glil-call> inst nargs
-This expression is probably misnamed, as it does not correspond to
-function calls. @code{<glil-call>} invokes the VM instruction named
address@hidden, noting that it is called with @var{nargs} stack arguments.
-The arguments should be pushed on the stack already. What happens to
-the stack afterwards depends on the instruction.
address@hidden deftp
address@hidden {Scheme Variable} <glil-mv-call> nargs ra
-Performs a multiple-value call. @var{ra} is a @code{<glil-label>}
-corresponding to the multiple-value return address for the call. See
-the notes on @code{mv-call} in @ref{Procedure Call and Return
-Instructions}, for more information.
address@hidden deftp
address@hidden {Scheme Variable} <glil-prompt> label escape-only?
-Push a dynamic prompt into the stack, with a handler at @var{label}.
address@hidden is a flag that is propagated to the prompt,
-allowing an abort to avoid capturing a continuation in some cases.
address@hidden, for more information.
address@hidden deftp
-
-Users may enter in GLIL at the REPL as well, though there is a bit
-more bookkeeping to do:
-
address@hidden
-scheme@@(guile-user)> ,language glil
-Happy hacking with Guile Lowlevel Intermediate Language (GLIL)!
-To switch back, type `,L scheme'.
-glil@@(guile-user)> (program () (std-prelude 0 0 #f)
-                       (const 3) (call return 1))
address@hidden 3
address@hidden example
+Tree-IL is a convenient compilation target from source languages.  It
+can be convenient as a medium for optimization, though CPS is usually
+better.  The strength of Tree-IL is that it does not fix order of
+evaluation, so it makes some code motion a bit easier.
 
-Just as in all of Guile's compilers, an environment is passed to the
-GLIL-to-object code compiler, and one is returned as well, along with
-the object code.
-
address@hidden Assembly
address@hidden Assembly
-
-Assembly is an S-expression-based, human-readable representation of
-the actual bytecodes that will be emitted for the VM. As such, it is a
-useful intermediate language both for compilation and for
-decompilation.
-
-Besides the fact that it is not a record-based language, assembly
-differs from GLIL in four main ways:
+Optimization passes performed on Tree-IL currently include:
 
 @itemize
address@hidden Labels have been resolved to byte offsets in the program.
address@hidden Constants inside procedures have either been expressed as inline
-instructions or cached in object arrays.
address@hidden Procedures with metadata (source location information, liveness
-extents, procedure names, generic properties, etc) have had their
-metadata serialized out to thunks.
address@hidden All expressions correspond directly to VM instructions -- i.e.,
-there is no @code{<glil-lexical>} which can be a ref or a set.
address@hidden Open-coding (turning toplevel-refs into primitive-refs,
+and calls to primitives to primcalls)
address@hidden Partial evaluation (comprising inlining, copy propagation, and
+constant folding)
address@hidden Common subexpression elimination (CSE)
 @end itemize
 
-Assembly is isomorphic to the bytecode that it compiles to. You can
-compile to bytecode, then decompile back to assembly, and you have the
-same assembly code.
-
-The general form of assembly instructions is the following:
-
address@hidden
-(@var{inst} @var{arg} ...)
address@hidden lisp
-
-The @var{inst} names a VM instruction, and its @var{arg}s will be
-embedded in the instruction stream. The easiest way to see assembly is
-to play around with it at the REPL, as can be seen in this annotated
-example:
-
address@hidden
-scheme@@(guile-user)> ,pp (compile '(+ 32 10) #:to 'assembly)
-(load-program
-  ((:LCASE16 . 2))  ; Labels, unused in this case.
-  8                 ; Length of the thunk that was compiled.
-  (load-program     ; Metadata thunk.
-    ()
-    17
-    #f              ; No metadata thunk for the metadata thunk.
-    (make-eol)
-    (make-eol)
-    (make-int8 2)   ; Liveness extents, source info, and arities,
-    (make-int8 8)   ; in a format that Guile knows how to parse.
-    (make-int8:0)
-    (list 0 3)
-    (list 0 1)
-    (list 0 3)
-    (return))
-  (assert-nargs-ee/locals 0)  ; Prologue.
-  (make-int8 32)    ; Actual code starts here.
-  (make-int8 10)
-  (add)
-  (return))
address@hidden example
-
-Of course you can switch the REPL to assembly and enter in assembly
-S-expressions directly, like with other languages, though it is more
-difficult, given that the length fields have to be correct.
-
address@hidden Bytecode and Objcode
address@hidden Bytecode and Objcode
-
-Finally, the raw bytes. There are actually two different ``languages''
-here, corresponding to two different ways to represent the bytes.
+In the future, we will move the CSE pass to operate over the lower-level
+CPS language.
 
-``Bytecode'' represents code as uniform byte vectors, useful for
-structuring and destructuring code on the Scheme level. Bytecode is
-the next step down from assembly:
address@hidden Continuation-Passing Style
address@hidden Continuation-Passing Style
 
address@hidden
-scheme@@(guile-user)> (compile '(+ 32 10) #:to 'bytecode)
address@hidden #vu8(8 0 0 0 25 0 0 0            ; Header.
-       95 0                            ; Prologue.
-       10 32 10 10 148 66 17           ; Actual code.
-       0 0 0 0 0 0 0 9                 ; Metadata thunk.
-       9 10 2 10 8 11 18 0 3 18 0 1 18 0 3 66)
address@hidden example
address@hidden CPS
+Continuation-passing style (CPS) is ...
 
-``Objcode'' is bytecode, but mapped directly to a C structure,
address@hidden scm_objcode}:
address@hidden Bytecode
address@hidden Bytecode
 
address@hidden
-struct scm_objcode @{
-  scm_t_uint32 len;
-  scm_t_uint32 metalen;
-  scm_t_uint8 base[0];
address@hidden;
address@hidden example
+Blah blah ...
 
-As one might imagine, objcode imposes a minimum length on the
-bytecode. Also, the @code{len} and @code{metalen} fields are in native
-endianness, which makes objcode (and bytecode) system-dependent.
-
-Objcode also has a couple of important efficiency hacks. First,
-objcode may be mapped directly from disk, allowing compiled code to be
-loaded quickly, often from the system's disk cache, and shared among
-multiple processes. Secondly, objcode may be embedded in other
-objcode, allowing procedures to have the text of other procedures
-inlined into their bodies, without the need for separate allocation of
-the code. Of course, the objcode object itself does need to be
-allocated.
-
-Procedures related to objcode are defined in the @code{(system vm
-objcode)} module.
-
address@hidden {Scheme Procedure} objcode? obj
address@hidden {C Function} scm_objcode_p (obj)
-Returns @code{#f} if @var{obj} is object code, @code{#f} otherwise.
address@hidden deffn
address@hidden File Format}
 
address@hidden {Scheme Procedure} bytecode->objcode bytecode [endianness]
address@hidden {C Function} scm_bytecode_to_objcode (bytecode)
-Makes a bytecode object from @var{bytecode}, which should be a
-bytevector. @xref{Bytevectors}.  By default, the embedded length fields
-in the bytevector are interpreted in the native byte order.
address@hidden deffn
+(system vm loader)
 
 @deffn {Scheme Variable} load-thunk-from-file file
 @deffnx {C Function} scm_load_thunk_from_file (file)
@@ -792,35 +524,7 @@ created for use in UNIX systems.  Guile has its own ELF 
linker and
 loader, so it uses the ELF format on all systems.
 @end deffn
 
address@hidden {Scheme Variable} write-objcode objcode file
address@hidden {C Function} scm_write_objcode (objcode)
-Embed object code into an ELF container, and write it out to a file.
-
-This procedure is part of a separate module, @code{(language objcode
-elf)}.
address@hidden deffn
-
address@hidden {Scheme Variable} objcode->bytecode objcode [endianness]
address@hidden {C Function} scm_objcode_to_bytecode (objcode)
-Copy object code out to a bytevector for analysis by Scheme.  By
-default, the length fields in the @code{struct scm_objcode} are
-interpreted in the native byte order.
address@hidden deffn
-
-The following procedure is actually in @code{(system vm program)}, but
-we'll mention it here:
-
address@hidden {Scheme Variable} make-program objcode objtable [free-vars=#f]
address@hidden {C Function} scm_make_program (objcode, objtable, free_vars)
-Load up object code into a Scheme program. The resulting program will
-have @var{objtable} as its object table, which should be a vector or
address@hidden, and will capture the free variables from @var{free-vars}.
address@hidden deffn
-
-Object code from a file may be disassembled at the REPL via the
-meta-command @code{,disassemble-file}, abbreviated as @code{,xx}.
-Programs may be disassembled via @code{,disassemble}, abbreviated as
address@hidden,x}.
+likewise load-thunk-from-memory
 
 Compiling object code to the fake language, @code{value}, is performed
 via loading objcode into a program, then executing that thunk with


hooks/post-receive
-- 
GNU Guile