Re: [Chicken-users] code snippet (suggested for the wiki) - and several newcomer questions

[F. Wittenberger]

Am Sonntag, den 17.08.2008, 20:11 +0200 schrieb Tobia Conforto:
> Thank you for this commented version!
> 
> I will raise my own "novice questions" if you don't mind.
> 
> > No matter how careful you are, switching to kernel mode and back is  
> > expensive in comparison to register arithmetic.
> 
> 
> Yes, but do Chicken mutexes and condition variables incur in a switch  
> to kernel mode at all?  It's not obvious they do, since we're not  
> using OS threads.

Chicken primitives don't switch to kernel mode.  This was meant as a
comment about pros and cons of user level threads vs. native threads.  

Native threads have a disadvantage here, since they need to switch to
kernel mode.  futex(2)'s come rather close, but in the contention case,
the kernel mode is still needed.

> > User level thread systems, as chicken provides one, schedule in a  
> > way, which could be understood as cooperative under the hood.  In  
> > chicken, any C expression is never interrupted.
> >
> > [[[ Is this actually true or just my understanding? ]]]
> 
> This is very interesting and I would like to know if it's true or not,  
> and what constitutes a "C expression" in this context.

Anything, which is plain C.  In this example the foreign-lambda*'s
string content.

I'm not sure how things work when C code calls back to chicken.

How know actually?  I intend to make a fuse module for chicken, but I
don't want to start before I know there's a chance to succeed.

> > (define string-ref++
> >   (foreign-lambda*
> >      char
> >      ((scheme-pointer buf) ((c-pointer integer) i))
> >      "char *p=(char *)buf; return(p[(*i)++]);"))
> 
> Why are you using return instead of C_return?  Is it intentional, or  
> just a mistake?

As far as I understand return(x) is mapped by chicken to C_return(x).
Since this form looks more like pure C, I prefer it.

So it's kind of taste.  Anybody how knows real reasons to choose one
form over the other?

> > Beginners did ask, why all these [x x]-bindings?  We keep a local  
> > reference in case the global one becomes redefined. This is a  
> > questionable practise.  While it's just what the doctor ordered...
> 
> Is this standard Chicken practice?  Why are you forbidding the user  
> (or another egg) to redefine these functions?  What is the purpose?

I'm following here the style seen in the chicken source, merely because
I'd like to get to know why it's used so much.  There could be a
performance reason (e.g. a chance for some optimiser to catch in???).
So probably only Felix knows a good answer.

Otherwise I wanted raise awareness among reader who wonder why the can't
screw with bindings as they might be used to from other Scheme systems.

> > (let ((mutex (make-mutex name))
> >       (condition (make-condition-variable name))
> >       (queue (make-queue))
> >       (buf #f))
> >   (define (eof?) (eq? #!eof buf))
> >   (define (buf-empty?) (or (not buf) (fx>= off (string-length buf))))
> 
> What does it mean for buf to be #f, #!eof, and ""?  Do all three mean  
> the same thing?  (I hope not.)

#f    : unknown -> check queue
#!eof : EOF in input port
string: buffer content

> Why is buf-empty? allowed to crash with an error when buf is #!eof?

Because it's not exported and to be used *only* if (eof?) failed
already.  No reason to do the same check twice.  ((I forgot in my last
message to state the reason why I'm using the undocumented arguments to
make-input-port: the code runs over pretty much all my network i/o.
When it was forced through the one-character-at-a-time interface, it
became a bottleneck))

> Also, at this point it's not clear to me why you have both a buffer  
> and a queue, although I'm sure it will become clear in a moment, so  
> maybe you should write a paragraph about the purpose of your variables.
> 
> >  (define (read-input!)
> >    (mutex-lock! mutex)
> >    (if (buf-empty?)
> >        (if (queue-empty? queue)
> >        (begin
> >          (mutex-unlock! mutex condition)
> >          (read-input!))
> >        (begin
> >          (set! buf #f)
> >          (set! buf (queue-remove! queue))
> >          (set! off 0)
> >          (mutex-unlock! mutex)))
> >        (mutex-unlock! mutex)))
> 
> So: if both buf and the queue are empty, it waits on the condition  
> variable; if buf is empty but the queue is not, it pops a string (or #! 
> eof, I guess) from the queue into buf; if buf is not empty, it does  
> nothing.  I'm starting to see what the queue and buf are for.
> 
> Why did you put (set! buf #f) just before another (set! buf)?

Because I know that the buf's size might be up to 256 Kbyte.  I wanted
to give the garbage collector an additional chance.

But you are right: queue-remove will not block, this should have been
done immediately when the last character has been consumed.  We should
delete that line.

> >  (define (read!)
> >    (if (eof?) buf
> >        (if (buf-empty?)
> >        (begin (read-input!) (read!))
> >        (string-ref++ buf (location off)))))
> 
> It would appear to me that this whole part should be inside your  
> critical section, as in read-input! and write!.  Am I missing your  
> point here?  I mean, even assuming string-ref++ is atomic in Chicken  
> threads (which still needs confirmation)

You can confirm that yourself.  Have a look at the generated C code.
Compare for instance library.c and tcp.c library.scm contains
(disable-interrupts) tcp.scm not.  In tcp.c there's a lot of
"C_check_for_interrupt;" macros right at the start of every C function.
Now compile string-ref++ and see where the interrupts are checked.

Or do it in your head: how the hell cold chicken interrupt the processor
without relying on native threads?  BTW: all user level thread systems
work similar.

>  you cannot hope to put this  
> whole if tree outside any mutex and don't run into trouble. The same  
> goes for ready?, read-string, and all the rest.

The test (eof?) could be outside the locked section.  The rest should go
in.  That's why I switched at that point to use (disable-interrupts) for
the whole unit.  Now there's no thread switches at all anymore (and thus
no point anymore to make string-ref++ atomic by implementing it in C; a
Scheme version would to as well).

/Jörg