Re: [libmicrohttpd] async answer to request

[Christian Grothoff]

On 12/04/2012 07:59 PM, Sebastiano Merlino wrote:
> Hi all,
> I am a big fan of the library and I realized several projects using it.
>
> I am now trying to write a proxy using libmicrohttpd and curl; at the 
> moment, I simply do a curl_easy_perform to the real service when I 
> receive a request. I am using an INTERNAL_SELECT mode to manage 
> requests with a pool of threads. I prefer to avoid 
> THREAD_PER_CONNECTION to avoid explosions in terms of thread number 
> and EXTERNAL_SELECT cause the limitation to a single thread.
>
> Whenever I receive a call I have to wait the real server response in 
> order to be able to give a response to the client, and when I have a 
> lot of concurrent requests the pool is exhausted and connections are 
> queued; so I am planning to use the curl multi interface to avoid the 
> blocking on curl request and use an async io system to enhance 
> parallelism.
>
> Is it possible to delay my response (from libmirohttpd) to the time 
> when I receive the response from curl without completely block one 
> thread of the pool?
>
> Am I completely blind and is there another and more simple system to 
> do what I am trying to do?

Well, the 'simple' answer is the one I believe you already have given: 
use 'EXTERNAL_SELECT' mode, which allows you to return 0 from the 
content reader callback to say "no data yet" and to integrate MHD with a 
curl-multi event loop.  This is what we do in an HTTP(S) socks proxy in 
GNUnet, so I can safely say that it does work.  THREAD_PER_CONNECTION 
and curl_easy should also work, and --- unless you are on an embedded 
system --- I'm not sure the number of threads is a major issue 
(especially as with any select-based approach, you're limited to 
FD_SETSIZE (typically 1024) sockets and thus <512 connections).  So if 
we set the same concurrency limit for the thread-based approach, we need 
512 threads, which is not all that scary given modern memory sizes on 
servers/desktops.

So anyway, as for scaling, I would suspect that the EXTERNAL_SELECT 
setup will first be limited by your FD_SETSIZE (unless you raised it) 
and not by the CPU being limited to a single thread.   So from an 
engineering point of view, I would urge you try that and only optimize 
further if you have established a need to do so via benchmarks.


Now, suppose you're on an embedded multi-core system with low-power 
cores, and your benchmarks tell you that multi-threading is needed; or 
maybe you do something else that is really expensive with the data or 
have some larger FD_SETSIZE. What to do in those cases?

Right now my answer would be to start multiple MHD-daemons, using the 
*same* listen socket.  If you open the listen socket yourself and pass 
it to MHD, you can have one listen socket be shared among multiple MHD 
daemon handles. So basically, you open one listen socket, start a bunch 
of threads, and have each of them run an external select loop around a 
per-thread MHD-daemon handle. This would essentially create an 'external 
select thread pool' mode. You will need to put the 'listen' socket into 
non-blocking mode (to avoid blocking one of the threads on a race for 
the socket --- note that the threads racing for the socket is _intended_ 
in this design, and the same race is used in the "internal select'-based 
thread-pool of MHD).

While I have not tried this 'external-select thread pool', I'm 
reasonably confident that it would work.  Still, if you do get it to 
work, I'm sure the list would appreciate an update ;-).


My 2 cents

Happy hacking!

Christian