Re: [Chicken-users] http egg and closed persistent connections

[Peter Bex]

On Mon, Feb 23, 2009 at 12:59:58AM -0800, Drew Hess wrote:
> Hi,
> 
> As you can see, I'm continuing to hack on the http egg. I've got one
> more patch coming after this one.

Keep 'em coming :)

> Some HTTP servers will drop a connection even when they've indicated
> it'll be persistent. For example, the nytimes.com web servers send a
> Connection: keep-alive header when the client sends an HTTP/1.0 request,
> but drops the connection after a single reply, anyway. http:send-request
> sees the keep-alive header and places the connection into the persistent
> connection pool. The next time http:send-request connects to
> nytimes.com, it'll re-use that connection if the user hasn't passed
> explicit in and out port parameters, but because the connection is
> closed, http:send-request's header variable gets EOF. Currently,
> http:send-request doesn't detect this condition, and it returns an eof
> object as the header value.
> 
> (It'd be nice if this behavior were documented in the wiki, by the
> way. The documentation says that the header value returned by
> http:send-request is always a string.)

This is not the intended behaviour, ie it's a bug and should be fixed
(as your patch does), so I don't see why it would need to be changed in
the docs.  Also, as John indicated, it's a public wiki :)  Feel free to
improve the docs.

> Programs that use http:send-request but ignore the in and out port
> values returned by http:send-request (e.g, when using the HTTP HEAD
> method) have no defense against this condition except _not_ to ignore
> in and out, to close them explicitly when http:send-request returns
> EOF, and then to re-try the request. That's a bunch of special-case
> code for a client app that just wants to send requests and doesn't
> care or even know about persistent connections.

Yeah, currently the http egg really doesn't abstract a whole lot of
protocol stuff.  Personally, I think HTTP got way too complicated but
it's one of the most popular protocols around nowadays so we'd better
try to get support for it right :)

[snip]

> A more backward-compatible way to deal with this error condition would
> be to signal (exn i/o net), so that programs that are already
> handling that exception will handle this new one, as well.

This sounds like a better solution. It's also more consistent with the
way other net I/O errors happen: the code already has provisions for
reconnecting when such an error occurs.

I've rolled in a patch that does exactly this.  Care to test if it
works for your particular usecase (I'm not quite sure how to reproduce
it)?  It's in the svn trunk.  I'll wait for the next patch you mentioned
before tagging a new release.

> p.s. All of the patches I'm sending are diffed against the current SVN
> trunk for the chicken-3 version of the http egg. None of the patches
> depends on any of the others. I can send a cumulative patch, too, if
> whoever's maintaining the egg wants to apply them all and it's easier
> that way.

The current patches are perfect; they can be tested and applied
individually.

Cheers,
Peter
-- 
http://sjamaan.ath.cx
--
"The process of preparing programs for a digital computer
 is especially attractive, not only because it can be economically
 and scientifically rewarding, but also because it can be an aesthetic
 experience much like composing poetry or music."
                                                        -- Donald Knuth

pgpyqRrZM7Yly.pgp
Description: PGP signature