Re: [libmicrohttpd] MHD_create_response_from_callback

[Christian Grothoff]

Regis, here is a patch along the lines I'm thinking of (also in SVN HEAD).

$ svn diff
Index: src/daemon/connection.c
===================================================================
--- src/daemon/connection.c     (revision 16779)
+++ src/daemon/connection.c     (working copy)
@@ -316,6 +316,11 @@
       MHD_destroy_response (pos->response);
       pos->response = NULL;
     }
+  if (pos->response != NULL)
+    {
+      MHD_destroy_response (pos->response);
+      pos->response = NULL;
+    }
 }
 
I suspect this might do the trick...

Happy hacking,

-Christian

On Tuesday, September 13, 2011 09:33:57 AM Regis Louge wrote:
> Calling 'still connected' from what would inevidably be another thread is
> not
> 
> > safe (as in, there is a race and you can't be sure that the 'connection'
> > handle is not freed in the mean time), so that's not a solution.
> > 
> > Can you motivate a bit better why you need this cleanup 'as fast as
> > possible'
> > (and what exactly your constraints are here?)?  Reading Eivind's
> > response, that would still not be 'as fast as possible', as 1s is still
> > like an eternity
> > for modern machines.
> 
> What I want to create is a push notification communication between my
> server and clients that would want to subscribe to that communication,
> indeed, when performing a GET on the URI "/push", a client would open this
> connection and receive push notifications of what is happening on the
> server. Those notifications depend on the other activities on the server
> (namely some PUT requests that it receives on other URI). Thus a client
> should be able to subscribe to these notifications but also to
> un-subscribe ! in the case of un-subscription, the client should also be
> able to resubscribe as soon as possible. Let's take for example a client
> crashing, when he recovers, he should be able to subscribe to this push
> notification communication like nothing ever happens. Thus you are right
> when you say that (at least for my case) 1s is an eternity.
> 
> > An alternative direction might be for us to call the connection cleanup
> > handler inside the thread and thus terminating the handler thread itself.
> > That way, you would know instantly the connection is "gone" and could do
> > your
> > own cleanup --- even though the MHD per-connection memory itself has not
> > been
> > completely cleaned up (that would still happen upon the next connection
> > inside
> > of the main thread, I see little alternative to that). Would that work?
> 
> I like the idea of knowing instantly that the connection is "gone" but I
> don't really get how you would do it. When exactly would you call the
> connection cleanup handler in the thread ?
> 
> > Happy hacking,
> > 
> > Christian
> > 
> > > On Fri, Sep 9, 2011 at 1:49 PM, Christian Grothoff
> > > 
> > > <address@hidden>wrote:
> > > > Hi Regis,
> > > > 
> > > > I've written a new testcase based on your specifications, and I'm now
> > > > even more
> > > > convinced that there is no bug.  The testcase will shortly be in SVN
> > > > HEAD. The test starts a MHD server (using most available threading
> > > > modes) and forks
> > > > a 'curl' process for the download. MHD then generates an "infiinte"
> > > > webpage of
> > > > "d" characters and the test driver eventually (after ~1s) kills curl
> > > > (SIGTERM).  The test driver then checks that the cleanup function is
> > > > indeed called.
> > > > 
> > > > Now, there is one *unusual* case which might have tripped you up
> > > > (certainly tripped me up when writing the testcase at first) in case
> > > > of the multithreaded
> > > > (one-thread-per-connection) MHD.  Here, the "cleanup" function is
> > > > only called
> > > > after MHD accepts the *next* inbound connection -- or upon MHD
> > 
> > shutdown.
> > 
> > > > The reason for this is simple: the master MHD thread sleeps until it
> > > > is awoken
> > > > from listen and only then cleans up 'left-over' connections. This,
> > > > potentially
> > > > 'untimely' cleanup is usually not an issue as it still avoids
> > > > unbounded (or even large) resource consumption.  However, if you
> > > > test with only a single connection and without MHD_daemon_stop, you
> > > > would think that MHD failed to call your cleanup function.  Just
> > > > remember this: the API guarantees that your
> > > > cleanup function will be called, but it makes no claims as to when
> > > > exactly this will happen exactly.
> > > > 
> > > > 
> > > > I hope this helps!
> > > > 
> > > > Happy hacking!
> > > > 
> > > > Christian
> > > > 
> > > > On Thursday, September 08, 2011 08:41:46 PM Regis Louge wrote:
> > > > > Here is my callback function :
> > > > > 
> > > > > static ssize_t
> > > > > push_callback (void *cls, uint64_t pos, char *buf, size_t max)
> > > > > {
> > > > > 
> > > > >     PushElement *push = cls;
> > > > >     if(push->pushReceived == 1){
> > > > >     
> > > > >         ...
> > > > >         Store push data in buf
> > > > >         ...
> > > > >         return strlen(buf);
> > > > >     
> > > > >     }
> > > > >     else return 0;
> > > > > 
> > > > > }
> > > > > 
> > > > > In my answer_to_connection for the URL /push I have :
> > > > > 
> > > > > response = MHD_create_response_from_callback (MHD_SIZE_UNKNOWN,
> > > > > 
> > > > >                                                          32 * 1024,
> >  
> >  &push_callback,
> >  
> > > > >                                                          push,
> > > > > 
> > > > > &push_free_callback);
> > > > > 
> > > > >            if(response == NULL)
> > > > >            {
> > > > >           
> > > > >           return MHD_NO;
> > > > >           
> > > > >            }
> > > > >            MHD_add_response_header (response, "Content-Type",
> > > > >            "text/html;
> > > > > 
> > > > > charset=utf-8");
> > > > > 
> > > > >            ret = MHD_queue_response (connection, MHD_HTTP_OK,
> > > > >            response); MHD_destroy_response (response);
> > > > >            return ret;
> > > > > 
> > > > > For a test case, when my client accesses to /push using curl -N
> > > > > -GET ".../push" he receives all the push notifications like it is
> > > > > suppose to, when I cancel it (^C), nothing happens on my server,
> > > > > request_completed is not called and neither is push_free_callback
> > > > > (which is the cleanup function)
> > > > > 
> > > > > On Thu, Sep 8, 2011 at 8:19 PM, Christian Grothoff
> > > > > 
> > > > > <address@hidden>wrote:
> > > > > > Hi!
> > > > > > 
> > > > > > Can you provide some kind of testcase that shows that MHD doesn't
> > > > > > properly call the cleanup function?
> > > > > > 
> > > > > > Happy hacking!
> > > > > > 
> > > > > > Christian
> > > > > > 
> > > > > > On Thursday, September 08, 2011 12:56:13 PM Regis Louge wrote:
> > > > > > > Hi,
> > > > > > > 
> > > > > > > I am currently trying to implement push notifications using
> > > > > > > libmicrohttpd and MHD_create_response_from_callback returning 0
> > > > > > > until an event triggers the push. This seems to work pretty
> > > > > > > fine except for when the client interrupts the communication,
> > > > > > > then the cleanup method is not called and thus creates a lot
> > > > > > > of problems and impossibility to "subscribe" again to the push
> > > > > > > stream.
> > > > > > > 
> > > > > > > --
> > > > > > > Regis
> 
> Regis