[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Chicken-users] Optional sqlite3 egg update
From: |
Thomas Chust |
Subject: |
Re: [Chicken-users] Optional sqlite3 egg update |
Date: |
Tue, 7 Mar 2006 14:51:42 +0000 (GMT) |
On Tue, 7 Mar 2006, JörgF. Wittenberger wrote:
[...]
Would you mind to share your experiences about inference with user level
threading?
Background: for Askemos (www.askemos.org a kind of p2p web server with
byzantine shared state of web apps) I wrote a sqlite3 binding to
rscheme. Now rscheme and chicken are quite simillar wrt. threading.
Therefore we've been very careful never to call any procedure of unbound
execution time from the scheme thread. [...]
The sqlite3 binding of mine (which is rough in comparision the your egg)
and the egg share this problem: calling sqlite3_step can potentially
take for ever. I'm not yet sure, but I feel that since I'm using
sqlite3 with askemos, it's sometimes suddenly unbearable unresponsive
and this seems to corellate with sqlite3 calls (probably autocommit).
Hello,
actually I must admit that I haven't used the SQLite3 egg for a real
multithreaded application so far. Nevertheless I have done some testing,
which suggests that everything works fine and the program stays responsive
under normal circumstances because the SQLite3 logic is indeed very fast
and in case a user defined function runs very long, the CHICKEN scheduler
does switch to other threads in the meantime.
In order to avoid memory leaks and messing up of the internal SQLite3
state, I do not allow user defined functions and collation sequences to
leave their original scope by invocation of continuations or signalling of
exceptions. This restriction seemed sensible to me and it is documented.
The use of SQL functions and collation sequences writen in Scheme of
course incurs a large performance penalty in comparison to ones written in
C, because the C<->Scheme boundary with all the wrapping and unwrapping of
data and the table lookups of function definitions and seed values have to
be gone through often.
Therefore I consider to throw much more C code in and associate a
pthread per sqlite3 struct, which would perform the actual calls and
call back to Scheme, when sqlite3 is done.
Do you think this is approach is useful and might be something to do
with the chicken egg too? Or would your experience suggest otherwise,
e.g. sqlite beeing so fast, we would rarely notice and I better go back
and find out where exactly those sudden delays come from.
Unfortunately I know nothing about the internals of RScheme, so I can't
really judge whether this is a good idea. I can imagine that it would make
several optimizations possible, but I can also imagine that in the end the
interthread communication and synchronization compensates fully for other
optimizations one could have made.
I definitely think that the scheme would be hard to implement for CHICKEN,
whose C API is not reentrant. As I am currently not too concerned about
speed problems with the sqlite3 egg, I will for the moment save myself the
effort of looking into this possibility.
As to the source of your sudden delays, some research where exactly they
come from may be useful...
[...]
--- sqlite3.scm~ 2006-03-06 22:33:56.000000000 +0100
+++ sqlite3.scm 2006-03-07 10:21:28.000000000 +0100
@@ -524,7 +524,7 @@
;; check whether the database is in autocommit mode
(define (sqlite3:auto-committing? db)
- (check-type 'sqlite3:auto-committin? db <sqlite3:database>)
+ (check-type 'sqlite3:auto-committing? db <sqlite3:database>)
((foreign-lambda bool "sqlite3_get_autocommit" sqlite3:database) db))
;; get the number of changes made to the database
Thank you for the patch, I applied it to the Subversion repository and
updated the egg on my homepage.
cu,
Thomas