libmicrohttpd
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [libmicrohttpd] MHD_create_response_from_callback


From: Christian Grothoff
Subject: Re: [libmicrohttpd] MHD_create_response_from_callback
Date: Thu, 15 Sep 2011 12:57:28 +0000
User-agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.1.16) Gecko/20110818 Icedove/3.0.11

Eh, MHD_connection_close is NOT part of the public API for good reasons. You MUST not use it, the symbol is not even exported by the library. Getting segmentation faults from using it yourself is expected. Do not do that.

Also, I am getting more an more confused about what you're trying to do. Naturally you CAN get two connections from the same source IP address at the same time, and your server should support that (otherwise it is simply broken). MHD has no problem handling more than one connection per IP at a time either.

I currently can only conclude that you're trying to do something with IP addresses that you should manage with sessions/cookies and that your application logic is most likely severely broken (based on not understanding TCP/IP and/or HTTP) and thus you're trying to do these strange things and come up with odd requirements like the 'instant' cleanup on connection closure.

TCP-FIN is not 'instant' and a browser may send us a TCP-SYN before we get the FIN, right? Not to mention, what would your system do with two users behind NAT using the same IP address?

With SVN HEAD, MHD is now calling the response destroy handler (as well as the 'connection completed' handlers) as soon as the OS tells us that the connection is dead. More cannot be done, and mucking around with MHD internals is not going to improve the situation. Also, you should NEVER EVER store a 'struct MHD_Connection' in your own data structures, that's virtually always a bug. Instead, store a pointer to your data structures in the 'void**' you're provided by MHD.

Happy hacking!

Christian

On 09/15/2011 09:15 AM, Regis Louge wrote:
Hi,

I have updated my version to the 0.9.14 that contains this patch
(without compilation error :) ). I managed to handle the disconnection
reconnection of the client by storing the MHD_Connection in my structure
and closing it manually whenever a call on "/push" is made from the same
IP address and a previous connection was stored :

if(client->connection != NULL) MHD_connection_close(client->connection,
MHD_REQUEST_TERMINATED_WITH_ERROR);
client->connection = connection;

Now I encounter something really weird, indeed, if a client disconnects,
I can "send" him a push notification without anything happening on my
web server, but when I "send" him a second one I have this weird
segmentation fault in :

int MHD_connection_handle_write (struct MHD_Connection *connection)
|
|_switch (connection->state)
      |
      |_case MHD_CONNECTION_CHUNKED_BODY_READY:
           |
           |_check_write_done (connection,
                             (connection->response->total_size ==
                              connection->response_write_position) ?
                             MHD_CONNECTION_BODY_SENT :
                             MHD_CONNECTION_CHUNKED_BODY_UNREADY);

It is right after the connection closes so connection->state is supposed
to be MHD_CONNECTION_CLOSED ?

Any idea what that would happen ? And why does the first push
notification attempt don't fail ?

Thanks a lot

On Tue, Sep 13, 2011 at 9:48 AM, Christian Grothoff
<address@hidden <mailto:address@hidden>> wrote:

    Regis, here is a patch along the lines I'm thinking of (also in SVN
    HEAD).

    $ svn diff
    Index: src/daemon/connection.c
    ===================================================================
    --- src/daemon/connection.c     (revision 16779)
    +++ src/daemon/connection.c     (working copy)
    @@ -316,6 +316,11 @@
           MHD_destroy_response (pos->response);
           pos->response = NULL;
         }
    +  if (pos->response != NULL)
    +    {
    +      MHD_destroy_response (pos->response);
    +      pos->response = NULL;
    +    }
      }

    I suspect this might do the trick...

    Happy hacking,

    -Christian

    On Tuesday, September 13, 2011 09:33:57 AM Regis Louge wrote:
     > Calling 'still connected' from what would inevidably be another
    thread is
     > not
     >
     > > safe (as in, there is a race and you can't be sure that the
    'connection'
     > > handle is not freed in the mean time), so that's not a solution.
     > >
     > > Can you motivate a bit better why you need this cleanup 'as fast as
     > > possible'
     > > (and what exactly your constraints are here?)?  Reading Eivind's
     > > response, that would still not be 'as fast as possible', as 1s
    is still
     > > like an eternity
     > > for modern machines.
     >
     > What I want to create is a push notification communication between my
     > server and clients that would want to subscribe to that
    communication,
     > indeed, when performing a GET on the URI "/push", a client would
    open this
     > connection and receive push notifications of what is happening on the
     > server. Those notifications depend on the other activities on the
    server
     > (namely some PUT requests that it receives on other URI). Thus a
    client
     > should be able to subscribe to these notifications but also to
     > un-subscribe ! in the case of un-subscription, the client should
    also be
     > able to resubscribe as soon as possible. Let's take for example a
    client
     > crashing, when he recovers, he should be able to subscribe to
    this push
     > notification communication like nothing ever happens. Thus you
    are right
     > when you say that (at least for my case) 1s is an eternity.
     >
     > > An alternative direction might be for us to call the connection
    cleanup
     > > handler inside the thread and thus terminating the handler
    thread itself.
     > > That way, you would know instantly the connection is "gone" and
    could do
     > > your
     > > own cleanup --- even though the MHD per-connection memory
    itself has not
     > > been
     > > completely cleaned up (that would still happen upon the next
    connection
     > > inside
     > > of the main thread, I see little alternative to that). Would
    that work?
     >
     > I like the idea of knowing instantly that the connection is
    "gone" but I
     > don't really get how you would do it. When exactly would you call the
     > connection cleanup handler in the thread ?
     >
     > > Happy hacking,
     > >
     > > Christian
     > >
     > > > On Fri, Sep 9, 2011 at 1:49 PM, Christian Grothoff
     > > >
     > > > <address@hidden <mailto:address@hidden>>wrote:
     > > > > Hi Regis,
     > > > >
     > > > > I've written a new testcase based on your specifications,
    and I'm now
     > > > > even more
     > > > > convinced that there is no bug.  The testcase will shortly
    be in SVN
     > > > > HEAD. The test starts a MHD server (using most available
    threading
     > > > > modes) and forks
     > > > > a 'curl' process for the download. MHD then generates an
    "infiinte"
     > > > > webpage of
     > > > > "d" characters and the test driver eventually (after ~1s)
    kills curl
     > > > > (SIGTERM).  The test driver then checks that the cleanup
    function is
     > > > > indeed called.
     > > > >
     > > > > Now, there is one *unusual* case which might have tripped
    you up
     > > > > (certainly tripped me up when writing the testcase at
    first) in case
     > > > > of the multithreaded
     > > > > (one-thread-per-connection) MHD.  Here, the "cleanup"
    function is
     > > > > only called
     > > > > after MHD accepts the *next* inbound connection -- or upon MHD
     > >
     > > shutdown.
     > >
     > > > > The reason for this is simple: the master MHD thread sleeps
    until it
     > > > > is awoken
     > > > > from listen and only then cleans up 'left-over'
    connections. This,
     > > > > potentially
     > > > > 'untimely' cleanup is usually not an issue as it still avoids
     > > > > unbounded (or even large) resource consumption.  However,
    if you
     > > > > test with only a single connection and without
    MHD_daemon_stop, you
     > > > > would think that MHD failed to call your cleanup function.
      Just
     > > > > remember this: the API guarantees that your
     > > > > cleanup function will be called, but it makes no claims as
    to when
     > > > > exactly this will happen exactly.
     > > > >
     > > > >
     > > > > I hope this helps!
     > > > >
     > > > > Happy hacking!
     > > > >
     > > > > Christian
     > > > >
     > > > > On Thursday, September 08, 2011 08:41:46 PM Regis Louge wrote:
     > > > > > Here is my callback function :
     > > > > >
     > > > > > static ssize_t
     > > > > > push_callback (void *cls, uint64_t pos, char *buf, size_t
    max)
     > > > > > {
     > > > > >
     > > > > >     PushElement *push = cls;
     > > > > >     if(push->pushReceived == 1){
     > > > > >
     > > > > >         ...
     > > > > >         Store push data in buf
     > > > > >         ...
     > > > > >         return strlen(buf);
     > > > > >
     > > > > >     }
     > > > > >     else return 0;
     > > > > >
     > > > > > }
     > > > > >
     > > > > > In my answer_to_connection for the URL /push I have :
     > > > > >
     > > > > > response = MHD_create_response_from_callback
    (MHD_SIZE_UNKNOWN,
     > > > > >
     > > > > >
      32 * 1024,
     > >
     > > &push_callback,
     > >
     > > > > >
      push,
     > > > > >
     > > > > > &push_free_callback);
     > > > > >
     > > > > >            if(response == NULL)
     > > > > >            {
     > > > > >
     > > > > >           return MHD_NO;
     > > > > >
     > > > > >            }
     > > > > >            MHD_add_response_header (response, "Content-Type",
     > > > > > "text/html;
     > > > > >
     > > > > > charset=utf-8");
     > > > > >
     > > > > >            ret = MHD_queue_response (connection, MHD_HTTP_OK,
     > > > > >            response); MHD_destroy_response (response);
     > > > > >            return ret;
     > > > > >
     > > > > > For a test case, when my client accesses to /push using
    curl -N
     > > > > > -GET ".../push" he receives all the push notifications
    like it is
     > > > > > suppose to, when I cancel it (^C), nothing happens on my
    server,
     > > > > > request_completed is not called and neither is
    push_free_callback
     > > > > > (which is the cleanup function)
     > > > > >
     > > > > > On Thu, Sep 8, 2011 at 8:19 PM, Christian Grothoff
     > > > > >
     > > > > > <address@hidden
    <mailto:address@hidden>>wrote:
     > > > > > > Hi!
     > > > > > >
     > > > > > > Can you provide some kind of testcase that shows that
    MHD doesn't
     > > > > > > properly call the cleanup function?
     > > > > > >
     > > > > > > Happy hacking!
     > > > > > >
     > > > > > > Christian
     > > > > > >
     > > > > > > On Thursday, September 08, 2011 12:56:13 PM Regis Louge
    wrote:
     > > > > > > > Hi,
     > > > > > > >
     > > > > > > > I am currently trying to implement push notifications
    using
     > > > > > > > libmicrohttpd and MHD_create_response_from_callback
    returning 0
     > > > > > > > until an event triggers the push. This seems to work
    pretty
     > > > > > > > fine except for when the client interrupts the
    communication,
     > > > > > > > then the cleanup method is not called and thus
    creates a lot
     > > > > > > > of problems and impossibility to "subscribe" again to
    the push
     > > > > > > > stream.
     > > > > > > >
     > > > > > > > --
     > > > > > > > Regis
     >
     > Regis




--
Regis




reply via email to

[Prev in Thread] Current Thread [Next in Thread]