Re: [Chicken-hackers] Numbers egg interaction with other compiled code.

[Tony Sidaway]

On 10/22/09, John Cowan <address@hidden> wrote:
> Tony Sidaway scripsit:
>
>> An alternative might be to provide chicken-install with the capability
>> to produce parallel egg binaries--one with numbers and one without--on
>> an egg-by-egg basis.
>
> That way lies a combinatorial explosion.  The utf8 egg is just like
> the numbers egg in this respect, only worse, because when a utf8-aware
> module passes a string to a vanilla module, the vanilla module *thinks*
> it understands it.  Boom, especially if the vanilla module mutates
> the string.

The way I see it, the combinatorial explosion is already with us, and
it is a result of the kind of Scheme implementation Chicken is.

Vanilla Chicken is fixnums, flonums.  R5RS compliant, very efficient
use of memory and processor power.  Other flavors of Chicken support
full numeric tower, UTF-8 characters, and so on.

Code units in Chicken are unaware of the different flavors of Chicken
code, and so they tend to load vanilla code by default.  Most eggs
will compile to a vanilla assumptions.  That's the problem.  I can
compile a single module to be bignum-aware (use numbers), but if a
dependency hasn't also been compiled to be bignum-aware then its
functionality may be unnecessarily crippled.

Right now I could probably resolve my problems by compiling the
SRFI-60 egg (which is the module I referred to as "fred" on Wednesday)
with numbers.  But that would slow down all the code written to work
with fixnums.  But isn't the bignum code aware that it needs to ask
for a bignum flavor of its dependencies?  It may not be able to obtain
such a flavor, but at least it could go looking for it.

And there also ought to be a capability for me to build a module with
a given flavor and store it on my system in a place separate from
other flavors, in a way that makes it accessible to code wanting to
use it.

And it seems to me that such a capability could be provided without
making life more complicated for those who don't mind if their code
returns odd errors or, sometimes, invalid results.

I suggest that we can help to resolve this by introducing flavor
declarations into our egg code.


Say I have a file in my chicken library called flavors, that says
something like:

(flavors
  (flavor bignum (priority 100) ((extension numbers)))
  (flavor utf-8 (priority 200) ((extension utf8 1.2))))

chicken could accept a switch "-flavor" which has the value of a
flavor listed in the flavors file--which is entirely under local
control.  Multiple uses of the switch, or values with comma-separated
list (I'm unsure of the conventions here) nominate one or more flavors
during compilation. A module compiled with a certain flavor can be
installed under library at ./FLAVOR1[/FLAVOR2]... for some ordering,
so a bignum and utf-8-flavored module can be placed at
LIBRARY/bignum/utf-8/MODULE.so

So if I tell csi I'm interested in loading bignum modules (I guess it
could also infer this when I tell it to use the "numbers" module), or
if I've compiled the main program with the "bignum" flavor, then
loading a given dependency consists of a search for appropriate files
under library path with the character string "/bignum/" in the path.
If the utf8 flavor is also requested then files under the library path
with the character string "/utf-8/" in the path are selected.  If the
intersection of these sets is non-null, then one of the files in the
intersection is loaded.  If the intersection is null, then a  file
satisfying flavors so as to minimised the sum of flavor priority
(assuming 100 means "more important than 200") is selected.  If none
of the flavors are matched then a vanilla version of the module is
loaded.

This is the kind of thing Scheme excels at and it would take place
while the application decides which compiled version of a file to load
to satisfy a dependency.  If no flavors are requested, nothing
happens. If the required flavors are not there, the load falls back to
vanilla.

For building, chicken and chicken-install can be adapted to
incorporate the appropriate declarations and compile to the
appropriate location.

The loader would be making decisions on the hoof so for debugging
purposes you'd need to be able to provide a  runtime trace of binary
files loaded.  You asked for bignum flavor modules, but did the bignum
version of module X get loaded?

So this is a little complex, but it wouldn't hit execution time or
build time for the vanilla flavor, and it would make it possible to
use Chicken for a much wider range of applications than is possible
now.  You could keep a bignum, utf-8 flavor of Chicken alongside a
vanilla flavor, on a module-by-module, egg-by-egg, program-by-program
basis.  This would enable us to extend support for the numerical tower
and other non-essential features that are actually very useful
sometimes.