Re: [libmicrohttpd] How much upload_data is uploaded to MHD_AccessHandlerCallback()?

[Kenneth Mastro]

For what it's worth, I ran into a similar/related problem using the built-in post processor.

Everything was working great, but then I created a web page with a large form on it.  One of the 'key/value' pairs was getting split up in 2 different calls to the callback function (i.e., I got half the value in one call, then the other half in a second call).  My post buffer was plenty big enough - it's just that the browser/OS decided to split it up (and/or had to because it exceeded the typical 1500-byte maximum packet size - not sure what the ultimate reason was).

Whatever the cause, I had to adapt my code to handle a 'value' that came as the result of 2 separate callback calls.  This wasn't terribly hard to do once I figured out that I needed to do it.

So - my advice is to go ahead and handle it.  If you care about efficiency you can probably consider it to be a 'rare event' which may help you streamline the 'full payload' case.  However, I wouldn't make the assumption that it'll always be all there.  (Which is kind of what Christian is getting at in his last sentence about the client doing something unusual.)

Ken

On Fri, Sep 11, 2015 at 1:58 AM, Christian Grothoff <address@hidden> wrote:

On 09/11/2015 01:56 AM, Bob Furber wrote:
> Can someone tell me how much upload_data is uploaded to
> MHD_AccessHandlerCallback()?

As much as is available at the time, except for chunked encoding. If the
client sends data with chunked encoding, then you may get each chunk
separately as we want to avoid having to memcpy().

> Can the upload_data uploaded to MHD_AccessHandlerCallback() ever be less
> than the lesser of all the data sent by the client or the size of the
> upload_data buffer?

It should generally be the MIN of those two, modulo the OS not making
all data available to MHD in a 'read/recv()' call and the possibility of
chunked encoding. (I think theoretically the OS could provide less data
than the full amount received for a TCP socket.)

So while in practice you are right about it, we don't make a formal
guarantee that this is true.

> In other words, if a POST request were to contain, say, 64 bytes and the
> upload_data buffer were larger, say, 256 bytes, could the upload_data
> buffer contain anything less than 64 bytes? Could one expect a short
> POST request to arrive in more than one packet?

The most crazy OS I'm aware of uses a TCP window of 1 byte and passes
TCP packets 1 byte at a time to the application, but there the OS has
technically not received (TCP-acknowledged) more than 1 byte at a time
either.

So this depends quite a bit on how portable you want to make your code.
Also, you may theoretically have a client that uses chunked encoding and
sends <64 bytes in the 1st chunk.