[Chicken-users] Functions with lots of parameters

[Alejandro Forero Cuervo]

In this email I discuss some problem I've been thinking about lately.
It has to do with how to write complex functions that receive lots of
parameters; I believe the conclusions on this discussion could have a
great impact on how maintainable and easy to call complex functions
are.  Hopefully others will share their thoughts and we will be able
to learn from each other's experiences.

As Scheme software grows in complexity, programmers will eventually
start writing functions that receive many parameters.  Most of the
time these will call other functions and also pass lots of parameters,
most being unmodified values they received.  Consider the following
code, which is a simplifed version of a real function in svnwiki:

    (define (get-toc-hook db path-in path-out path path-out-real linktypes 
extensions)
      (let ((toc (get-props-parents-first "svnwiki:toc" path-in path)))
        (if toc
          (render-file-contents db path-in path-out toc linktypes extensions 
(or path-out-real path))
          stream-null)))

    (define (render-file-contents db path-in path-out path . rest)
      (let-optionals rest ( ... )
        ...))

Notice how, after a few checks, {{get-toc-hook}} just ends up calling
{{render-file-contents}} with pretty much the same values it was
passed (the only exception being that it adds just one parameter at
the end).  As the complexity of the software grows,
{{render-file-contents}} will very likely start wanting even more
parameters... they will have to be passed to {{get-toc-hook}} and all
similar ''wrapper'' functions that call it.

Furthermore, notice that the last three parameters to
{{render-file-contents}} are optional, having default values.  We
would obviously want the last two parameters to {{get-toc-hook}} to be
optional as well, but in order to do this (using either keywords or
let-optional), the expresions that construct their default values
would need to be copied to {{get-toc-hook}}.

This situation is, obviously, far from ideal: code becomes verbose and
redundant as lots of functions start receiving many parameters merely
to pass them to the functions they call; in our example, if we ignore
the call to {{render-file-contents}}, of the 7 parameters it receives,
{{get-toc-hook}} only uses 2!  What's a programmer to do?

I will discuss the alternatives I can think of: structures and
environments.  keywords and parameters, by the way, have been
frequently brought up in conversations I've held with other
programmers about this topic, but they seem to change things very
little or bring complicated problems of their own, so I don't really
consider them worth discussing.

Lets start with structures.

One obvious alternative is using structures to hold parameters.  Our
{{get-toc-hook}}, in this fashion, would probably look as follows:

    (define-record config db path-in path-out path path-out-real linktypes 
extensions)

    (define (get-toc-hook config)
      (let ((toc (get-props-parents-first
                   "svnwiki:toc"
                   (config-path-in config)
                   (config-path config))))
        (if toc
          (let ((new-config (config-copy config)))
            (config-path-set! new-config toc)
            (config-path-out-real-set! new-config (or path-out-real path))
            (render-file-contents new-config))
          stream-null)))

This is indeed a very frequent practice in OOP programming, where one
finds things like:

    (let ((window (make-window)))
      (window-title-set! window "foo")
      (window-position-set! window 20 20)
      (window-size-set! window 100 200)
      (window-contents-add! window box)
      window)

While this does solve the above problem (for example, if
{{render-file-contents}} needs a new value, it can be simply included
in the record and we won't have to modify {{get-toc-hook}}), it does
pose new problems.

What happens when we make a new version of {{render-file-contents}},
call it {{render-file-contents-auth}}, that expects a {{username}} and
{{password}} values and ignores the {{linktypes}} and {{extensions}}
values?

One option is to create a new separate config-auth record.  This is
bad because we'll end up with lots of separate records, which will
make it difficult to call these functions.  Suppose, for example, that
{{render-file-contents}} somehow obtains values for the {{username}}
and {{password}} parameters and wants to call the variation function;
it will have to build a {{config-auth}} record, which is probably
going to end up being very verbose.  Since you'll end up with one
different record for each function that requires a “bundle” of
parameters (that is, for any function complex enough to benefit from
using a record, as opposed to receiving its parameters directly), I
suppose this is the worst and most difficult to maintain option.

The other option is extending the {{config}} record and marking some
of the fields as optional: if you call {{render-file-contents}},
{{username}} and {{password}} are ignored, whereas if you call its
variation, {{linktypes}} and {{extension}} are ignored.  This is only
marginally better, because you end up with half-complete structures
being passed around all the time.  That doesn't sound too good.

That's not the only problem of the above approach.  {{get-toc-hook}}
has to ''clone'' the record, since you want your function to be
referentially transparent with regards to the config records it gets
passed; that is, the caller of {{get-toc-hook}} should be allowed to
assume the {{config}} object it passed wasn't change (so he can reuse
it for other calls).  This isn't terribly bad, but I think
environments are a slightly better alternative.

Reconsider this whole problem under an hypothetic dynamically scoped
version of Scheme.  It seems -much- easier to solve, does it not?  Our
{{get-toc-hooks}} function would probably become something like this:

    (define (get-toc-hook)
      (let ((toc (get-props-parents-first "svnwiki:toc"))
            (path-out-real (or path-out-real path)))
        (if toc
          (render-file-contents)
          stream-null)))

So while having a lexical scope is a great characteristic for a
programming language to have (since it allows us to understand the
code far more easily than we would be able to understand it
otherwise), it does come with its price, which is the whole problem
we're discussing in this post: when the “environment” is very complex,
having to list all 20 parameters to any function can take the fun out
of programming.  And the whole point of programming is to have fun,
right?

If using a structure, as described in the former section, feels like
programming in a dynamically-scoped programming language it's because,
in a sense, it is.  For example, using a structure forces us to always
give the same name to each “field” in all our functions, just like
programming in a dynamically-scoped Lisp would.

By the way, I'm not advocating a dynamically scoped Scheme.

If you take a step back and look at the two previously described
approaches, ---explicitly passing functions all the arguments they
need or using structures to hold arguments and passing them around---,
you'll see in both cases we are modifying an “environment”.  In the
first approach we're doing it implicitly, modifying the Scheme-level
environment in which the body of the functions gets evaluated and the
“cloning” takes place whenever we make a function call (this cloning,
is, thankfully, an optimized operation, extending the environment); in
the second we're somehow capturing the environment inside a record
(and only cloning it when we need to make a change, as in the case of
{{get-toc-hook}}, which probably happens rarely).

So, as I mentioned, another option is to use explicit {{environment}}
objects, as opposed to implicit environments or explicit records.
Ideally, these should be mapped to the low-level objects used by our
implementation to represent regular environments (ie. operations on
them should be fast).  We should also export some useful convenience
functions/macros to create new empty environments; extend them; lookup
symbols on them.  With these functions the idea of using a record to
hold the values would allow us to rewrite {{get-toc-hook}} as:

    (define (get-toc-hook config)
      (let ((toc (get-props-parents-first "svnwiki:toc" (lookup config 
'path-in) (lookup config 'path))))
        (render-file-contents
          (extend ((toc toc)
                   (path-out-real (or (lookup config 'path-out-real)
                                      (lookup config 'path))))
            config))
        stream-null))

(It can be argued that {{lookup}} should be a macro and the second
expression it receives shouldn't appear quoted in the code.  I don't
want to discuss that right now.)

While this might seem alien to Scheme programmers, similar things are
very often done in other programming languages.  In Perl, for example,
this is often done with hashes.  In the end, it is pretty much the
same as using structures, the only differences being that in
structures you have to declare all the fields you will be using and
you can't normally "extend" structures (you have to make copies of
them).

A similar alternative, though probably not as good as using
environments, is, obviously, using alists or hash tables.

Something that would be nice to have would be a way to get an explicit
handle for the current environment; and, difficult (OoOoO!), a way to
call a procedure extracting values for its parameters from a given
environment based on the name they are given in the lambda form.
These two would make it easy to go from lexically scoped (ie. passing
values explicitly) to “explicitly dynamically scoped” (ie. passing
environments containing the values) at the programmers convenience.

Anyway, that's what I've thought about.  I'm not claiming its the best
solution; I'm not even claiming it would even work, I haven't actually
sat down to think how one would implement this.  I just wanted to
share my thoughts on this problem that seems to always hit me when my
Scheme programs grown in complexity beyond a certain basic level and
seem to make it increasingly difficult to keep working on them.  How
do others tackle this problem?

Alejo.
http://azul.freaks-unidos.net/