[libmicrohttpd] Re : Re: Re : Re: External select mode limited to 1024 FDs

[louisbenoit]

Le 11/03/15, Christian Grothoff  <address@hidden> a écrit :
> On 03/11/2015 12:58 PM, address@hidden wrote:
>> Hi Christian,
>>
>> I wanted to use external threads because my system has the following constraints:
>> 1) One connection per thread: the response to the client is time critical and I cannot afford that a connection delays another one.
>
> Then you should use the one-thread-per-connection mode, that is exactly
> what you want.  MHD will use a thread for each connection, which matches
> your requirements.  This method also has no limits on open FDs as MHD
> doesn't use select() but blocking read/write operations instead.

Are you refering to MHD_USE_THREAD_PER_CONNECTION? Or using MHD_OPTION_CONNECTION_LIMIT set to MHD_OPTION_THREAD_POOL_SIZE?
In both cases shouldn't I use MHD_USE_POLL in order not to use select ()?

>
>> 2) Many listening sockets are opened, hence many MHD daemon: some for HTTP and some for HTTPS.
>> 3) Up to 2000 concurrent connections (shared between HTTP and HTTPS).
>
> Well, 2000 isn't much, so one-thread-per-connection should be fine.
>
>> As I don't know how the traffic will be shared amongst the daemons, I would need to configured each MHD daemon with epoll and 2000 threads.
>> This solution does not scale. Just imagine the number of threads created at start up or debugging a single HTTP request with gdb.
>
> 2000 threads is fine on modern systems, and if you really run into
> problems, you can easily switch to thread-pool-with-epoll.  As for
> debugging, well, that's always an issue if you have 2000 concurrent
> actions ;-).
>
>> The best solution would be to share the worker threads pool between the daemons but if I understand correctly the current implementation does not allow it.
>
> Well, if you use on thread per connection, there is no sharing anyway.
> If you have HTTP and HTTPS, you are right right now you'd have two
> daemons and no joint pool. Two solutions: you can setup a reverse proxy
> using Apache or nginx to forward HTTPS to HTTP, or just live with a
> slight imbalance. I suspect the reverse proxy is the best option, as
> then you can even move that to another physical machine easily, and
> you're off-loading HTTPS.
>
>> This why I fall back on external thread: I create 2000 daemons for each listening socket and dispatch the HTTP/HTTPS request to a pool of 2000 threads, which reuse the daemons.
>>
>> Is there a plan to implement worker threads sharing between daemons? If not, are there other MHD solutions that I miss?
>
> I don't have a plan to do that, seems a bit overengineered.  For this
> stuff, I'd rather keep it simple unless given hard proof that this is a
> major issue.  I don't think without having performance measurements one
> should even worry about this.
>
>
> Happy hacking!
>
> Christian
>
>> Thanks,
>>
>> Louis
>>
>> Le 05/03/15, Christian Grothoff  <address@hidden> a écrit :
>>> Hi!
>>>
>>> I had no plans for an 'external poll' API, as we have epoll() which is
>>> better for very large FDsets (as long as you're on Linux). Also,
>>> redefining FD_SETSIZE is also IMO a very good possibility.
>>>
>>> Regardless, select/poll with > 1024 FDs and external mode is likely not
>>> the greatest idea from a scalability/performance perspective: if you
>>> have that many concurrent / open FDs, you really want epoll() and
>>> possibly think about using threads / thread pool for processing.
>>>
>>> Happy hacking!
>>>
>>> Christian
>>>
>>> On 03/04/2015 10:22 PM, address@hidden wrote:
>>>> Hi,
>>>>
>>>> I would like to use the external select mode, as I need to manage the connection threads externally.
>>>> Because there could be much more than 1024 FDs used at run time, I cannot use MHD_get_fdset() and select ().
>>>> Apart from redefining FD_SETSIZE, are there other ways to use external threads when FD > 1024?
>>>> If not, is there a plan to add an MHD_get_poll_event API, the poll version of MHD_get_fdset()?
>>>>
>>>> Thanks,
>>>>
>>>> Louis Benoit
>>>>
>>>
>>>
>>>
>>