Re: [Chicken-users] help please

[Alex Shinn]

At Sun, 11 Apr 2004 09:58:02 -0400, Linh Dang wrote:
> 
> > I'm sure we can tune this a good deal. For example the list-ref's in
> > parse-ctags-line look a bit crude.  Heavy compiler-optimizations
> > won't give anything here, since mnost of the work is done in the
> > libraries.
> 
> Would you please give abit more explanations. I'm a scheme newbie
> absolutely clueless wrt the chicken implementation.

I'd recommend a parser instead of a regexp, since it's a simple enough
pattern.  Perl regexps are heavily optimized, and because they use
special variables will be hard to beat with a library matcher that
builds and deconstructs lists.

The biggest problem, however, is that you're using strings with a
default eq? hash-table which is certainly an error:

     #;> (hash-table-set! tab "foo" 1)
     #;> (hash-table-ref tab "foo")
     #f

This means your table is going to have multiple entries for the same
string, greatly increasing memory usage and causing algorithmically more
work when you send the results to the database, which is probably why
you see the order of magnitude decrease in speed.  You can fix this
either with string=? hash-tables:

     (define idents (make-hash-table string=?))

or by interning keys with string->symbol before putting them in the
table (try both and see which is faster).

Some other optimizations:

  1) If the current version you bind many variables in a let when the
     first line may not even succeed.  Instead of

     (let* ((tokens (string-match ...))
            (ident (and tokens ...))
            ...))

     you can try

     (let ((tokens (string-match ...)))
       (when tokens
          (let ((ident (and tokens ...))
                 ...))))

     or use and-let* if you want each term to be true.

  2) If using your own parser you may or may not get better performance
     reading chars or strings rather than lines.

> > Note that Perl is heavily tuned for these kinds of jobs... ;-)
> 
> I know but it's a write-only language.

Worse than that, it's a human-write-only language :)

-- 
Alex