Re: [Bug-wget] http server responding with 416 but file was not transferred completely

[Tim Rühsen]

Hi Josef,

pushed your patch - thanks for your contribution.

BTW, the example from your original post doesn't show that wrong server
behavior any more. Maybe the server got fixed meanwhile !?


With Best Regards, Tim



On 09/14/2017 07:47 PM, Josef Moellers wrote:
> On 14.09.2017 17:06, Tim Rühsen wrote:
>> On 09/14/2017 12:11 PM, Josef Moellers wrote:
>>> On 14.09.2017 10:12, Tim Rühsen wrote:
>>>> On 09/14/2017 09:53 AM, Josef Moellers wrote:
>>>>> Hi,
>>>>>
>>>>> We have seen (at least) one server who has
>>>>> Accept-Ranges: bytes
>>>>> in his header but, upon receiving a request with
>>>>> Range: bytes=23068672-
>>>>> responds with
>>>>> HTTP/1.1 416 Requested Range Not Satisfiable
>>>>> although the file was not transferred completely!
>>>>>
>>>>> wget then claims that
>>>>> The file is already fully retrieved; nothing to do.
>>>>>
>>>>> E.g.
>>>>> run
>>>>>   wget https://downloads.dell.com/FOLDER02721216M/1/SUU_14.12.200.69.ISO
>>>>> the, after a couple of MB, abort the transfer and then continue the
>>>>> download:
>>>>>   wget --continue
>>>>> https://downloads.dell.com/FOLDER02721216M/1/SUU_14.12.200.69.ISO
>>>>>
>>>>> Maybe the check in src/http.c:
>>>>> 3821   if (statcode == HTTP_STATUS_RANGE_NOT_SATISFIABLE
>>>>> 3822       || (!opt.timestamping && hs->restval > 0 && statcode ==
>>>>> HTTP_STATUS_OK
>>>>> 3823           && contrange == 0 && contlen >= 0 && hs->restval >= 
>>>>> contlen))
>>>>>
>>>>> should be changed and any HTTP_STATUS_RANGE_NOT_SATISFIABLE with an
>>>>> incomplete file should show something like
>>>>>
>>>>> "download continue failed, file incomplete"
>>>>
>>>> Well, that would be ok for this special server.
>>>>
>>>> Normally 416 together with a server timestamp matching the file's
>>>> timestamp means that the file is complete (as far as the client can
>>>> judge from HTTP).
>>>>
>>>> IMO, if the server is broken (or misbehaves) then the server should be
>>>> repaired. Except it is a very common misbehavior. In which case we could
>>>> consider a work-around on the client side.
>>>>
>>>
>>> So I humbly propose the attached patch.
>>> I tried to create a pull request, but got a 403.
>>
>> Thanks for the patch - I'll test it in the next days.
> 
> I have attached a simple webserver that simulates the error:
> when a request with a Range comes in, it replies with 416 and also
> returns an unsanely huge Content-Length. You'll need glib2 and
> microhttpd for it to build.
> 
> I was able to reproduce the issue with this server and check that the
> patch fixes it by causing wget to retry (until --retries is exhausted).
> 
>> BTW, we currently work on Wget2 where we have a related issue, if you
>> like to take a look at it: https://gitlab.com/gnuwget/wget2/issues/278
> 
> I'll do that.
> 
> Josef
>

signature.asc
Description: OpenPGP digital signature