Re: [libmicrohttpd] ContentReaderCallback has changed behaviour in latest releases

[Christian Grothoff]

On 01/31/2013 05:43 PM, David J Myers wrote:
> Hi Christian,
>
> I have a problem with the latest releases of libmicrohttpd
>
> Previously I was using version 0.9.7 and I have recently upgraded to 
> 0.9.20 and I have also checked this in 0.9.24
>
> I use a ContentReader Callback to chunk my video data
>      
> response = MHD_create_response_from_callback (MHD_SIZE_UNKNOWN,
>
>                                                 1024,
>
> &crc, responseptr, &crcf);
>
> Sometime, in the callback I throw away a frame of data, so I return 
> zero from the callback, like this.
>
> static ssize_t
> crc (void *cls, uint64_t pos, char *buf, size_t max)
> {
>
> .
>
> .
>
> .
>
> if (NothingToDo)
>
> {
>
>     return 0; // MHD should call me back later
>
> }
>
> else
>
> {
>
>   //do stuff
>
> .
>
> .
>
> .
>
>     return no_of_bytes_processed;
>
> }
>
> }
>
> If I return zero from the callback I expect MHD to call me back some 
> short time later.
>
> This has always worked for me in the past but now, when I return zero, 
> from the callback, version 0.9.20 does not call me back until after a 
> delay of around 6 seconds by which time the client has timed out. 
> Version 0.9.24 is worse and actually crashes the program with the 
> following error:-
>
> Fatal error in GNU libmicrohttpd connection.c:395: API violation
>
>                                                                 Aborted
>
> Is it no longer allowed to return zero from a ContentReaderCallback?

That depends on your threading mode.  From the documentation:

/**
 * Callback used by libmicrohttpd in order to obtain content.  The
 * callback is to copy at most "max" bytes of content into "buf".  The
 * total number of bytes that has been placed into "buf" should be
 * returned.<p>
 *
 * Note that returning zero will cause libmicrohttpd to try again,
 * either "immediately" if in multi-threaded mode (in which case the
 * callback may want to do blocking operations) or in the next round
 * if MHD_run is used.  Returning 0 for a daemon that runs in internal
 * select mode is an error (since it would result in busy waiting) and
 * will cause the program to be aborted (abort()).
 *
 * @param cls extra argument to the callback
 * @param pos position in the datastream to access;
 *        note that if an MHD_Response object is re-used,
 *        it is possible for the same content reader to
 *        be queried multiple times for the same data;
 *        however, if an MHD_Response is not re-used,
 *        libmicrohttpd guarantees that "pos" will be
 *        the sum of all non-negative return values
 *        obtained from the content reader so far.
 * @param buf where to copy the data
 * @param max maximum number of bytes to copy to buf (size of buf)
 * @return number of bytes written to 'buf';
 *  0 is legal unless we are running in internal select mode (since
 *    this would cause busy-waiting); 0 in external select mode
 *    will cause this function to be called again once the external
 *    select calls MHD again;
 *  MHD_CONTENT_READER_END_OF_STREAM (-1) for the regular
 *    end of transmission (with chunked encoding, MHD will then
 *    terminate the chunk and send any HTTP footers that might be
 *    present; without chunked encoding and given an unknown
 *    response size, MHD will simply close the connection; note
 *    that while returning END_OF_STREAM is not technically
 *    legal if a response size was specified, MHD accepts this
 *    and treats it just as MHD_CONTENT_READER_END_WITH_ERROR;
 *  MHD_CONTENT_READER_END_WITH_ERROR (-2) to indicate a server
 *    error generating the response; this will cause MHD to simply
 *    close the connection immediately.  If a response size was
 *    given or if chunked encoding is in use, this will indicate
 *    an error to the client.  Note, however, that if the client
 *    does not know a response size and chunked encoding is not in
 *    use, then clients will not be able to tell the difference between
 *    MHD_CONTENT_READER_END_WITH_ERROR and 
MHD_CONTENT_READER_END_OF_STREAM.
 *    This is not a limitation of MHD but rather of the HTTP protocol.
 */


So returning 0 is illegal if you're using MHD's internal select mode, as 
we don't want applications to do busy-waiting.  It is quite possible 
that we changed the library behavior here a bit, to perform more 
stringent checking.  There was a change related to that logic between 
0.9.13 and 0.9.14.

If you can tell me what threading mode you're using (or even have a 
small example), that would be helpful to find out more about what's 
going on.

Happy hacking!

Christian