Re: [Chicken-users] Re: compiling jbogenturfa'i .scm => .c takes 2.5 hours

[Alan Post]

On Sat, Nov 27, 2010 at 09:55:27AM -0700, Alan Post wrote:
> On Sat, Nov 27, 2010 at 04:05:01PM +0100, Felix wrote:
> > From: Alan Post <address@hidden>
> > Subject: [Chicken-users] Re: compiling jbogenturfa'i .scm => .c takes 2.5 
> > hours
> > Date: Fri, 26 Nov 2010 15:16:54 -0700
> > 
> > > 
> > > I would love to better understand what it is I'm doing and why it is
> > > causing a problem.  I'm not sure how to fix my code, as I don't
> > > quite get why what I'm doing is triggering the compiler to generate
> > > such large files.
> > 
> > Alan, if you use "-debug 2", the compiler will output the result of
> > the macro-expansion, which may be helpful in looking at the code
> > your macros generate. Somewhere excessive code is generated, which
> > may be less costly in the interpreter due to sharing of subexpressions.
> > 
> 
> I've rerun the build with "-debug 2", and it sent a lot of output to
> the terminal.  The build hasn't finished running yet, but the
> terminal output has stopped.  Here is the file so far, which I think
> is what you were asking for.  If this file is too much, I'll run the
> same command on a smaller portion of the grammar (say just the
> morphology file) or on a toy grammar.  It should still be showing
> the problem, just perhaps not at the scale I see on the full Lojban
> grammar.
> 
> Looking at this, can you point me in the right direction? (172K):
> 
>   http://lodockikumazvati.org/download/jbogenturfahi.build.log.bz2
> 
> I traced statement45, which is behaving roughly like I would expect,
> with it being set to undefined, used in several lambda expressions,
> and then defined inside the letrec.  The same story with sumti75.
> Is it the excessive number of ##core#lambda expressions that is causing
> my problem?  If that isn't the right track, nothing here is catching
> my eye and so I begin to expect something post-macro expansion.
> 
> Thank you,
> 

Reviewing the output from -debug 2, and rereading R5RS's
documentation on letrec, I've reworked the compiler and moved
the lambda that was being created for each use of a letrec variable
to the definition: '(record (lambda () ...)' rather than (record ...).
I changed the appearance of 'record' in other letrec definitions to
'(record)' rather than '(lambda () record)'.

Essentially, I'm definining a lambda for each element in the letrec
(~880 elements) rather than for each usage (many, many more than
880).

The file now compiles in minutes rather than hours, and it is
roughly 7MB in size, rather than 61MB.

I haven't checked this change in yet, as my left/right recursion tests
now fail and I broke any non-terminal where memoization is turned off:
This change was a proof of concept that I now need to clean up.

However, things are looking good.  I'll report back after I've
cleanup up and checked in all the code to confirm that everything is
working.  So far, this does look like the culprit.

-Alan
-- 
.i ko djuno fi le do sevzi