Re: [libmicrohttpd] Server-Sent Events

[Christian Grothoff]

On Monday 23 May 2011 14:55:26 Peter Ross wrote:
> Hi,
> 
> Server-Sent Events (SSEs) are a form of 'push technology', where the server
> leaves the connection open indefinitely, and sends response data to the
> web-browser as those events happen.
> 
> Standard: http://www.w3.org/TR/eventsource/
> Example:  http://googlecodesamples.com/html5/sse/sse.html
> 
> Implementing events in the current release of MHD isn't very practical.
> The only way it can be achieved is via MHD_create_response_from_callback,
> where the reader callback blocks until an event is ready to be sent.
> This only works with MHD_USE_THREAD_PER_CONNECTION.

You're missing one scenario.  You can also have the callback return 0 in 
*external select* mode instead of blocking.  Then, you need to find a way to 
"unblock" the external select yourself (by putting some of your own FDs or a 
lower timeout into the call) and then call MHD again.  MHD will repeatedly 
call the callback to pull for more data then (you're free to continue to 
return 0).  However, MHD would NOT go into busy waiting because after you 
return 0, it will NOT put the socket of the connection into the write-set for 
the external select.
 
> How could the MHD library and API be improved to better handling of SSEs
> and other streaming approaches. Has anyone else encountered this problem?

Now, the above is how the API is supposed to work, I don't know if anyone has 
done this already (and the tutorial / testcases / examples are also not 
covering this case yet either).  But, at least in theory, the API and 
implementation support this even if you are not using threads.


Finally, there are two possible issues left: (1) you cannot use either method 
with 'internal select' or the thread pool and (2) anytime the external select 
(or poll) wakes up for any reason, your callback will be invoked again; this 
might become a performance issue if you're talking doing this with 100,000 
clients in parallel that have mostly idle connections at the same time. 

Both of these issues could theoretically be solved with an explicit "unblock" 
API call (so the callback would return 0 and only be called again after the 
client explicitly re-enables the connection; even connection timeouts would be 
disabled in the meantime); a new flag would be used to require the unblock call 
to maintain binary compatibility.  Again, I'm not sure this is needed, but if 
you require solutions to (1) or (2) a nice patch (nice == with documentation, 
tutorial, testcases...) along those lines would likely be accepted.


Happy hacking!

Christian