Re: [libmicrohttpd] recv MSG_DONTWAIT and EAGAIN

[Christian Grothoff]

Given that we defined MSG_DONTWAIT to 0 if it was not defined by the platform, clearly the logic was never supposed to depend on non-blocking operations. I think the DONTWAIT was added as a "conservative" flag ("just in case"), after all, the calls are all guarded by 'select' and 'poll'.

So this makes the situation here interesting:

1) select/poll should only allow us to get to recv if data is ready (and note

that we're using select/poll even with threads). So technically the

DONTWAIT should not have mattered --- dead code should go, right?

2) On the other hand, the DONTWAIT may have prevented worse -- a blocking

read of the server due to some other error that would stop all

connections was avoided at the expense of dropping one.

Clearly neither is good. Furthermore, we got an EAGAIN which, while unexpected (see above) it by itself usually not a reason to close a connection --- except if our select/poll is broken and we're about to go into a busy-waiting spin cycle, maybe then closing the connection is a good idea after all?

Finally, based on your report, the issue is tricky for you to reproduce ("timing related"), so I don't have an easy testcase to settle who is to blame (after all, theoretically your application could have read data from the socket between MHD's select/poll and recv call, or corrupted memory, or ... --- not that I'm saying this is even likely, my money is on poll+threads still having some deeper trouble that I just cannot find by looking at the code).

Given all this, I've decided to follow your suggestion and remove MSG_DONTWAIT (from both recv and send calls) for now, the primary reason being that it was never used on *all* platforms and so this should give us consistent behavior, if nothing else, and, by design, it should do no harm. [SVN 14672]

I'll keep the code to close connections on EAGAIN as well, after all, that should not have happened and now definitively should never happen in the first place.

Naturally, I'd be very interested to hear if someone now suddenly observes blocking behavior on a recv call (obviously hard to observe on pthread-per-connection...) even more so if they can produce a testcase to demonstrate it.

Happy hacking,

Christian

On Wednesday, March 16, 2011 07:25:05 pm Eivind Sarto wrote:

> I am using libmicrohttpd-0.9.8 for a prototype video streaming application

> and I am seeing some unexpected error messages from the library when I run

> the application on a very fast server. I think the behavior is timing

> related, because it does not occur when I run the same code on a slightly

> slower server, or if I enable an MHD_OPTION_URI_LOG_CALLBACK handler on

> the fast server.

>

> In daemon.c:recv_param_adapter(), RECV(MSG_DONTWAIT) will frequently return

> EAGAIN, and this causes connection.c:do_read() to call

> connection_close_error().

>

> Is this the right logic?

> Calling RECV(MSG_DONTWAIT) should expect EAGAIN if there is no data queued

> on the socket head. It is not really and error that should close the

> connection.

> What do you think?

>

> I am running the library with flags = MHD_USE_THREAD_PER_CONNECTION |

> MHD_USE_POLL, so for debug I decided to try and remove the MSG_DONTWAIT

> flag. And the application would then run without any errors.

>

> Note: I do not have any control over the client. I am streaming live video

> to an Apple handheld device.

>

> -eivind