Re: [Chicken-users] Async IO (was Re: libcurl?)

[Peter Keller]

On Tue, Oct 04, 2005 at 10:21:55PM -0000, Thomas Chust wrote:
> CHICKEN does not have any feature that would make continuations  
> serializable; it would mean that you somehow have to transform machine  
> pointers to arbitrary memory locations where your continuations' assembler  
> code lives into stuff that can portably be transferred over a network  
> connection.

I'm sure there are some ideas that could help you perform a virtual
serialization of the continuations. For example, machine A has the
"original" copy of the running executable, machine B has a certain form
of a skeleton executable running in memory with a very small memory foot
print. Now, the continuation (wholesale) is transferred to the target
machine where it is propped up in the some executable pages in the same
range as before and control jumps there.  Of course, the rest of the
memory heap (known by the skeleton since it conspired with machine A)
is protected via mprotect, and when the first segfault happens, it is
caught, and the offending pages brought in from machine A to machine
B. Then the continuation happily continues. After a while, a working
set emerges, and machine B's heap gets managed independantly of machine A.

Machine A knows what pages got served, so it is aware of such things
during garbage collection, and when machine B's pages are finished,
they are merged back into Machine A's domain. I'm handwaving the merge
process, since I haven't thought about this more than 5 minutes. Though I
suspect the merging algorithm can move stuff out of the pages of machine
B to better location on machine A since ultimately you're dealing with
higher level objects and the scheme implementation isn't going to care.

I'm sure there are a lot more gotchas in a method like this (like race
condition between garbage collection on machine A and machine B's request
for the pages), especially without deep thinking about it, but it might be
workable.

> If you get this working, I honestly have to congratulate you -- I tried it  
> once (trying to cook up something like Kali Scheme on CHICKEN basis) but  
> failed miserably to come up with any decent, let alone portable or even  
> elegant solution. The day anyone has a really brilliant idea how to solve  
> the problem, I will certainly resume this dead project of mine -- it would  
> be a wonderful toy ;)

Did you happen to document the methods you tried?

-pete