[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Memory leak in unix/pthread.fs
From: |
nature |
Subject: |
Memory leak in unix/pthread.fs |
Date: |
Tue, 08 Nov 2022 15:38:54 +0100 |
User-agent: |
mu4e 1.8.11; emacs 28.2 |
Hi,
I am working on a web server written in pure Forth and I am using gforth
as the implementation I develop on. Serving multiple requests without
blocking requires me to venture into the realm of multi-threading and
gforth's experimental support for pthread has been a blessing. It may
be good for me to mention that I work on UNIX (OpenBSD).
Now instead of doing like in cilk.fs and spawning a few workers and
offloading requests to them, I prefer to create a separate thread each
time a request comes and this led me to notice that my server wasn't
able to serve more than ~3k requests before hanging and also the memory
usage of my process increased a lot with each new request comming in. I
then proceeded to hunt down memory leaks due to the S" S\" and S+ words
with minimal impact. That is what lead me to realize that creating new
threads with the PASS word was creating non-detached thread which never
freed the memory once the thread exited.
I am attaching a patch for unix/pthread.fs which fixed the big memory
leak for me, my server is no able to server well above 3k requests
without hanging/crashing (I tested 10 without any issues).
--- pthread.fs.old Tue Nov 8 15:22:26 2022
+++ pthread.fs Tue Nov 8 15:22:56 2022
@@ -247,7 +247,7 @@
r> swap >r save-task r@ 's !
1+ dup cells negate sp0 r@ 's @ -rot sp0 r@ 's +!
sp0 r@ 's @ swap 0 ?DO tuck ! cell+ LOOP drop
- pthread-id r@ 's 0 thread_start r> pthread_create drop ; compile-only
+ pthread-id r@ 's pthread_detach_attr thread_start r> pthread_create drop ;
compile-only
: pass ( x1 .. xn n task -- ) \ gforth-experimental
\G activates task, and passes n parameters from the data stack
Also optionally I am attaching another patch for cilk.fs just to
correctly get the number of cores on OpenBSD.
--- cilk.fs.old Tue Nov 8 15:31:33 2022
+++ cilk.fs Tue Nov 8 15:32:43 2022
@@ -20,7 +20,7 @@
require unix/pthread.fs
-e? os-type s" darwin" string-prefix? [IF] s" sysctl -n hw.ncpu"
+e? os-type 2dup s" darwin" string-prefix? -rot s" openbsd" string-prefix? or
[IF] s" sysctl -n hw.ncpu"
[ELSE] s" nproc" [THEN]
r/o open-pipe throw slurp-fid s>number drop 1 max Value cores
Thank you for the great work on gforth, I can't wait for the official
1.0 to release so that OpenBSD ports won't be stuck with 0.7.3 forever! :D
Cheers,
--
nature
- Memory leak in unix/pthread.fs,
nature <=