Re: [libmicrohttpd] libmicrohttpd: keep-alive connection alternative?

[Christian Grothoff]

On Monday 25 August 2008 12:28:46 pm Andreas Røsdal wrote:
> Hello!
>
> I'm using libmicrohttpd as a HTTP server, and wonder how I should
> configure libmicrohttpd optimally for the way I am using it.
> I think that HTTP keep-alive connections support would be perfect,
> but since that appears to be unspported I wonder about alternatives.

Unsupported by what?  MHD should support keepalive / persistent connections. 
Obviously your proxy may not support them, but MHD should.

> libmicrohttpd is setup to be a HTTP server in my back-end process.
> A single web browser connects to libmicrohttpd through a HTTP proxy
> based on the Apache Commons HttpClient. A libmicrohttpd server process
> always connects to a single web browser client, because of the proxy.
> As a result, the proxy always connects to a single libmicrohttpd process,
> where about 4 connections every second is established between the proxy
> and libmicrohttpd process. So a a huge amount of connections are created
> and closed for the communication between the proxy and libmicrohttpd.
>
>        [Web browser]  <->  [Proxy] <-> [libmicrohttpd]
>
> I suspect in my current config, the connections are not closed and cleaned
> up fast enough, and therefore uses more resources than neccesary.
> If I set MHD_OPTION_CONNECTION_TIMEOUT = 1, then the connections are
> cleaned up fast, but then the CPU usage goes to over 50% (way too high).
> If I set to MHD_OPTION_CONNECTION_TIMEOUT = 0 (no timeout), then the
> memory usage increases a lot.
>
> So if the same connection could be reused or kept-alive, then I'd hope
> that performance would be greatly improved. So do I need HTTP keep-alive
> support, or is there anything I could do to improve my situation?

Using timeouts does not seem to be the right approach here (they are mostly to 
ensure that totally inactive clients don't hog the server indefinitely).  If 
you *want* the TCP connections to be cut after each exchange, you could 
proably add an explicit header "Connection: Close" (check HTTP spec for 
spelling) to your response.  Then MHD should close the connection immediately 
after the response has been transmitted completely.

In either case, I am not sure I understand why with 4 requests/second and a 
timeout of 1s you would get 50% CPU load (short of some bug in MHD or your 
code or rather expensive response generation code).  In my tests, 300-500 
requests/second (on loopback) were no problem.

Christian