Hi,
At the end of every HTTP request to be made with url-http.el and containing a body, an unnecessary "\r\n" is appended, and additionally those two characters are not used in the calculation of the Content-Length header. This normally would not matter, because a carefully build server will anyway only read "Content-Length" bytes from the body and ignore the final CRLF, but Emacs additionally defaults to using Connection: keep-alive, which results in the TCP traffic for what was meant to be a single request, being interpreted as two separate HTTP requests, the first one being roughly the intended one, and the other one consisting only of CRLF. In particular, I am using the HTTP server from net.http in Go language. That keepalive is enabled by default is strange, especially given how the variable that controls this is described:
(defvar url-http-attempt-keepalives t
"Whether to use a single TCP connection multiple times in HTTP.
This is only useful when debugging the HTTP subsystem. Setting to
nil will explicitly close the connection to the server after every
request.")
Those issues have been somewhat discussed here, but it seems the people discussing unfortunately don't understand HTTP:
https://groups.google.com/forum/#!msg/gnu.emacs.bug/SF4P7gVI6IQ/SExtWzutKI4J
- a newline terminating the sequence of headers
- an optional request body
The request body will be read by the server only if one of the headers is a Content-Length header, and the value of the header is exactly the number of bytes that is being sent, there is no CRLF terminating the body. So if there is no body, the request ends with two CRLFs, if there is a body, it just ends with [Content-Length] bytes of raw data. There is no possibility a proper HTTP server could be confused by a request being terminated with two CRLFs, if the request is otherwise correct. I think there must have been some confusion as to the reason of the original problem, that was then turned into this "fix".
For reference, my code is roughly this, and as mentioned I am using net.http from the Go language on the other end:
(defun server-command (command)
(let* ((url-request-method "POST")
(url-request-extra-headers '(("Content-Type" . "application/x-www-form-urlencoded")))
(url-request-data (concat (url-hexify-string "command") "=" (url-hexify-string (json-encode command))))
(result-buffer (url-retrieve-synchronously server-url))
(result
(with-current-buffer result-buffer
(goto-char (point-min))
(delete-region (point-min) (search-forward "\n\n"))
(buffer-string))))
(kill-buffer result-buffer)
result))
Normally I get from this function the contents of the first, "correct" response body from the server, but if I run it a few times in quick succession I additonally get the string "HTTP/1.1 400 Bad Request" at the end, which is actually the second HTTP response showing up at random in the buffer (altough it's consistently sent by the server every time, as I can see in a sniffer).