Re: [Lynx-dev] Bug#673452: lynx: does not handle all Content-Encodings advertised in Accept-Encoding

[Thomas Dickey]

On Fri, May 18, 2012 at 07:17:39PM +0000, Thorsten Glaser wrote:
> Dominik George dixit:
> 
> >lynx advertises in its Accept-Encoding header sent to HTTP servers that it
> >accepts the "compress" Content-Encoding.
> >
> >Upon receiving actual content with Content-Encoding: compress, it 
> >thoughtlessly
> >passes it to gunzip, which obviously cannot handle compress (.Z) data and 
> >gives
> >up saying it cannot decompress the stream.
> 
> We’ve talked about this in IRC, and I checked the actual data
> returned by the site: http://m.faz.net
> 
> I analysed the content, and it was most definitely *not* in
> the “compress” scheme, which I’ve looked up is defined as the
> output of the original Unix compress(1) programme – which I
> happen to have in the MirBSD base system.
> 
> Moreover, I thought it was headerless, but prefixing a standard
> compress(1) header \x1F\x9D\x90 did not help. (Using \x1F\x9D\x10
> made it decompress to garbage.)
> 
> Then I guessed that the sender was wrong and looked actual
> algorithms for generating such content up, and found one,
> in php, in the minify Google code project. It uses gzcompress()
> which most notably does *not* use compress but deflate/zlib
> encoding.
> 
> Putting \x1F\x8B\x08\0\0\0\0\0 (an empty gzip header) in front
> of the received data, et voilà, it decompressed with gzip(1).

If it's a widely-used defective function, we could always provide a workaround 
:-)

The usual problems in this area are (defective) servers which send gzip'd
data for deflate (I recall 2-3 instances).  A server which strips the headers
from the compressed data makes things a little more difficult.
 
> So, the server is at fault here for sending bogus data.
> 
> I also did the check the other way ’round: compress(1) a file
> on MirBSD with the original Unix compress, scp it to a Debian
> box, run it through gunzip from the GNU gzip package, read the
> result, it matches.

gunzip is designed to replace uncompress (in addition to its more usual 
functionality)
(gzip does not replace compress - they really are different)

-- 
Thomas E. Dickey <address@hidden>
http://invisible-island.net
ftp://invisible-island.net

signature.asc
Description: Digital signature