Re: [libmicrohttpd] Some issues with MHD

[Christian Grothoff]

Hi!

Ok, so I've solved most of the mystery.  SVN 36568 was indeed wrong, the
pool reallocation was correct before 0.9.45 and will be again right in
0.9.46 (to be uploaded ASAP).  I now put a comment as to why it was OK
to begin with.

Eugenio: your 'strlen(line)' change is also in 0.9.46, as the code
didn't tolerate "extra" empty lines in the protocol, but it is clearly
better to not drop a connection just because between two requests there
is an unexpected empty line. (Especially since the spec maybe unclear
with respect to how many \r\n one should put at the end of a POST
with/without a footer).  So that makes MHD more tolerant, which is a
good thing (and harmless).

What I didn't quite figure out is why there is then still this
buffer-shrinkage that motivated the SVN 36568-change to begin with: I
went again over the code (now with more understanding), and couldn't
find anything.  So for that, I would now really appreciate a test that
demonstrates this.

Anyway, happy hacking!

Christian

On 11/03/2015 07:08 PM, Eugenio Perez wrote:
> Hi All.
> 
> First of all, congrats for this excellent library! I think it does
> have a clear interface and it's very usable.
> 
> However, I have found two issues. I don't know if I'm using bad the
> library, so please correct me if I'm wrong.
> 
> I'm trying to send, using one connection, a lot of small (~2K) POST
> messages (in the real case, JSON text). If I use not chunked encoding
> messages (i.e., specifying Content-Length header), all is OK.
> 
> However, I've found that if I try to send the messages in chunks, MHD
> drops the connection, and the client needs to reconnect again. Testing
> in deep, I've found that:
> 
> * If I increase MHD_OPTION_CONNECTION_MEMORY_LIMIT, the connections
> are dropped slower (i.e., the same connection last longer).
> * If I send smaller messages that the sent in http-post.c, MHD does not drop.
> * If I change URL length, drops rate changes:
>   * ./http-post "http://10.0.70.175:8080/aaaaaaaaaaaaaaaaaaaaaaaaaaaa";
> causes drops
>   * ./http-post
> "http://10.0.70.175:8080/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"; no drops
> * If I apply the patch attached, MHD does not drop my connection
> anymore. However, I don't know if it affects some other MHD internals.
> 
> All of this happens with MHD 0.9.44. I've seen that you just announced
> 0.9.45. However, if I update is even worse: connections are being
> dropped almost every packet or 2 packet! If I bisect, the commit that
> introduce the issue is r36568 (with message "-fix assertion failure
> from race on shutdown and buffer shrinkage with pipelining").
> 
> I attach client and server I'm using to test situation. In order to
> detect reconnection, I just use:
> 
> tcpdump -nni any "port 8080 and tcp[tcpflags] & (tcp-syn) != 0"
> 
> Where 8080 is the post I'm using to testing.
> 
> Am I using the bad options? Is the patch valid to fix the issue? if
> not, what could I do to keep debugging this?
> 
> Thanks and regards.
>