Re: [libmicrohttpd] Memory leak and global variables

[Nicolas Mora]

Hi Kenneth, thanks a lot for your answer, you helped me moving on with 
my problems.

I fixed a lot of small problems thanks to your answers, but the memory 
leaks are still here with the same conditions: about 20kb is lost each 
time I call the 6 main webservices simultaneously with jquery, but no 
leak at all when I call each webservice on by one.

I'm sure I'll find the solution, I'm so close.

Here are my feedbacks.

> * I'm far from an expert, but are you sure you can use
> thread-per-connection and use-select-internally together?  (You very
> well may be able to, I just thought they weren't normally used
> together.  Could be completely wrong about that, though.)
Not really, maybe it's not recommended, but the program never yelled at 
me for that. I skipped select-internally flag, to keep just 
thread-per-connection, but no obvious change since then.

> * There's a minor bug in the last release (0.9.37 and earlier) of MHD
> where it doesn't properly cycle through the existing connections when in
> thread-per-connection mode.  The fix was simple (essentially deleting
> one character), but if you're detecting leaks by looking at the result
> just before your program terminates, there's a remote chance that has
> something to do with it.
Actually, I'm using raspian repository, which provides a 0.9.20-1 
release. I tried yesterday using the last release by downloading, 
compiling and installing it, but not change then too.

> * I haven't noticed any leaks in MHD.  I'm not saying there aren't any,
> but I haven't seen anything like what you're describing.  (Granted - I'm
> not hammering it like you would be in a web-service-based solution.)
I'm pretty sure the leaks come from my code somewhere, but most likely 
because of my misuse of MHD, that's why I'm asking the questions here.

> * You have a lot going on in that code.  If it were me, I'd strip it way
> down to a simple test case to try and isolate the leak(s).  You're doing
> a lot of mallocs in general - any chance you could (at least
> temporarily) remove them and reply with fake data?  The point is to try
> and see if the problem is memory that MHD is allocating or if it's
> something you're doing.
You're right about the useless mallocs. I changed all the fixed size 
mallocs to local variables, there is one (the page variable) that can't 
be change to a local variable since its size may vary.
I'll try with fake data as you suggest it.

> * You're doing a few mallocs at the start of the request - are you sure
> that's necessary (e.g., for the 'page' and 'urlcopy')?  Also, are you
> sure they're all getting cleaned up?  For testing, maybe you could just
> create a stack-based character string (i.e., no malloc) and tell MHD to
> copy the response string instead of de-allocating the memory later.  I'm
> not sure that would be any less efficient, anyway - malloc is kind of
> slow, and your strings are really short.  (I.e., instead of 'MUST_FREE'
> use the one that tells MHD to copy the buffer).  I mean - you're
> returning in the middle of the code all the time (MHD_YES, MHD_NO), but
> your cleanup code executes at the end of the function.  That seems very
> very suspicious to me.  In fact - I bet that's what your problem is (or
> at least one of them).  That connection request function gets called
> multiple times - but you'll return early some times and the cleanup code
> at the end never gets executed.  That leaves your 'urlcopy' (and
> possibly other things) not freed, right?
You were totally right, I forgot about those return statements, and the 
'free' calls that were forgotten there.

> * I see you have a 'request completed' callback that frees the memory
> you allocated for/in your connection info.  I assume you've verified
> that it's getting called?
Yes, the request_completed function is called at the end of each 
angharad_rest_webservice function.

> * As for the globals - I didn't analyze your code in any kind of depth,
> but you probably already know that MHD can take a pointer to any object
> you want and keep giving you the pointer through the connection cycle
> (typically a structure).  At a glance, it looks like you're taking
> advantage of that.  For things that you need across connections.. I'm
> not sure what the suggested usage pattern is.  I'm using C++, so I just
> use class data members.
I finally found it, indeed the parameter dh_cls in MHD_start_daemon is 
designed for that ! :)

> That's all I've got.  Hope it helps.
That helped, thanks

/Nicolas