Re: [Lynx-dev] Bug#673452: lynx: does not handle all Content-Encodings advertised in Accept-Encoding

[Thorsten Glaser]

Dominik George dixit:

>lynx advertises in its Accept-Encoding header sent to HTTP servers that it
>accepts the "compress" Content-Encoding.
>
>Upon receiving actual content with Content-Encoding: compress, it thoughtlessly
>passes it to gunzip, which obviously cannot handle compress (.Z) data and gives
>up saying it cannot decompress the stream.

We’ve talked about this in IRC, and I checked the actual data
returned by the site: http://m.faz.net

I analysed the content, and it was most definitely *not* in
the “compress” scheme, which I’ve looked up is defined as the
output of the original Unix compress(1) programme – which I
happen to have in the MirBSD base system.

Moreover, I thought it was headerless, but prefixing a standard
compress(1) header \x1F\x9D\x90 did not help. (Using \x1F\x9D\x10
made it decompress to garbage.)

Then I guessed that the sender was wrong and looked actual
algorithms for generating such content up, and found one,
in php, in the minify Google code project. It uses gzcompress()
which most notably does *not* use compress but deflate/zlib
encoding.

Putting \x1F\x8B\x08\0\0\0\0\0 (an empty gzip header) in front
of the received data, et voilà, it decompressed with gzip(1).

So, the server is at fault here for sending bogus data.

I also did the check the other way ’round: compress(1) a file
on MirBSD with the original Unix compress, scp it to a Debian
box, run it through gunzip from the GNU gzip package, read the
result, it matches.

So Lynx actually has no bug here: putting the content through
gunzip is the correct thing to do. Therefore closing the report
with Cc to upstream.

bye,
//mirabilos
-- 
FWIW, I'm quite impressed with mksh interactively. I thought it was much
*much* more bare bones. But it turns out it beats the living hell out of
ksh93 in that respect. I'd even consider it for my daily use if I hadn't
wasted half my life on my zsh setup. :-) -- Frank Terbeck in #!/bin/mksh