Re: [libmicrohttpd] Cannot set Content-Length in header

[Evgeny Grin]

Just to keep it in records.

I found at least two applications trying to use the flag 
MHD_RF_INSANITY_HEADER_CONTENT_LENGTH to set the "Content-Length" header 
for HEAD responses.

This is not a correct approach.

Application must provide *correct* response for head request. *Correct* 
means (in this context) including full response body. MHD will not send 
the response body when replying the HEAD requests, but will set correct 
"Content-Length" header in the reply body.
If response is created with zero size body, then reply for HEAD request 
(and for any other request) will have "Content-Length: 0" header.
If response is created by MHD_create_response_from_callback() with size 
MHD_SIZE_UNKNOWN then "Transfer-Encoding: Chunked" will be added instead 
of the "Content-Length" header.

In any case, the response for HEAD request should be created in the same 
way as for GET requests. MHD never sends response body if HTTP protocol 
requires no response body in replies.

The next MHD version (0.9.78) will allow creating special response 
object just for HEAD requests. Check sources in the official git repo if 
your are curious.
https://git.gnunet.org/libmicrohttpd.git/tree/src/include/microhttpd.h

--
Evgeny

On 25.10.2019 15:46, Christian Grothoff wrote:
> Hi Tim,
>
> In Git head, you can now do:
>
> MHD_set_response_options (response,
>                            MHD_RF_INSANITY_HEADER_CONTENT_LENGTH,
>                           MHD_RO_END);
>
> *before* calling MHD_add_response_header() to get your desired insanity ;-).
>
> Test for MHD_VERSION 0x00096702.
>
> Happy hacking!
>
> Christian
>
> On 10/25/19 11:40 AM, Tim Rühsen wrote:
> > Hi Christian,
> >
> > > Alternatively, we could define a bit-field
> > > option SANITY_CHECK to just disable certain sanity checks. WDYT?
> >
> > I like this better since we can have an easy start and slowly evolve it
> > with more flags / bits, as the need arises.
> >
> > Currently, the wget test suite only needs a flag to switch off the
> > Content-Length sanity checks. But I see more potential in the future -
> > e.g. we tests with weird / insane chunked transfer encoding, which
> > might be sanitized by a future MHD change.
> >
> > Regards, Tim
> >
> > On 10/25/19 10:56 AM, Christian Grothoff wrote:
> > > Hi Tim,
> > >
> > > I didn't realize this was wget-*testing* related.
> > >
> > > For your use-case, I agree that having an equivalent of the
> > > MHD_OPTION_STRICT_FOR_CLIENT like MHD_OPTION_STRICT_FOR_SERVER where you
> > > can set a "-1" to "allow application to break the protocol" could be
> > > reasonable.
> > >
> > > I'd prefer having such a more "generic" option over a specific one to
> > > just disable header checks. Alternatively, we could define a bit-field
> > > option SANITY_CHECK to just disable certain sanity checks. WDYT?
> > >
> > > Happy hacking!
> > >
> > > -Christian
> > >
> > > On 10/25/19 9:48 AM, Tim Rühsen wrote:
> > > > Hi Christian,
> > > >
> > > > for Wget2 we have to test and prepare for all kinds of malicious and
> > > > misconfigured / misbehaving servers. So what you call a new feature is
> > > > from our point of view a regression, since it breaks tests.
> > > >
> > > > Of course I understand your intention. But maybe we can have both by
> > > > adding a new option to switch off the Content-Type checks ?
> > > > Or a more general approach - a HEADER_CHECKS_OFF mode that make MHD just
> > > > being "dumb" ?
> > > >
> > > > Regards, Tim
> > > >
> > > > On 10/24/19 7:42 PM, Christian Grothoff wrote:
> > > > > Hi!
> > > > >
> > > > > The MHD documentation explicitly says that MHD does NOT allow the
> > > > > application to set the content-length header at all. You're likely
> > > > > ignoring an error code you are getting back from the library when trying
> > > > > to set the content-length header.
> > > > >
> > > > > Also, not allowing applications to break the HTTP protocol is a feature,
> > > > > not a bug. (That feature was indeed introduced around the versions you
> > > > > mention ;-).)
> > > > >
> > > > > Happy hacking!
> > > > >
> > > > > Christian
> > > > >
> > > > > On 10/24/19 6:25 PM, Archit Pandey wrote:
> > > > > > Hello all,
> > > > > >
> > > > > > I'm working on the test-suite of wget2 that uses libmicrohttpd.
> > > > > >
> > > > > > I've noticed that on v0.9.66 MHD when I try setting an arbitrary value
> > > > > > for Content-Length in the HTTP header, MHD changes it to the correct
> > > > > > value. This behavior is not present on 0.9.59. Additionally, I could not
> > > > > > find an option to revert to the previous behavior.
> > > > > >
> > > > > > Could this be a possible bug?
> > > > > >
> > > > > > --
> > > > > > Archit Pandey
> > > > > > Junior Year B.Tech.
> > > > > > Department of Computer Science and Engineering
> > > > > > National Institute of Technology Karnataka
> > > > > > Surathkal, India

OpenPGP_0x460A317C3326D2AE.asc
Description: OpenPGP public key

OpenPGP_signature.asc
Description: OpenPGP digital signature