Re: bug#23750: 25.0.95; bug in url-retrieve or json.el

[Yuri Khan]

On Fri, Dec 2, 2016 at 9:24 PM, Eli Zaretskii <address@hidden> wrote:

>> +    (when (mutibyte-string-p request)
>> +      (error "Multibyte text in HTTP request: %s, please translate any
>> multibyte components to unibyte using `encode-coding-string'" request))
>>       (url-http-debug "Request is: \n%s" request)
>>       request))
>
> This will also reject pure-ASCII strings that just happen to be
> multibyte, although there will be no problem with such an HTTP
> request.  Do we really want to disallow that use case?

It is really unfortunate that we talk about ASCII strings, unibyte
strings, multibyte strings, as if that was a meaningful
classification.

The real dichotomy is between text (aka strings) and MIME-type-tagged
byte arrays. In order to send a string over HTTP, one must encode it
to a byte array and tag it as "text/plain; charset=utf-8" or
"text/html; charset=utf-8" or application/json (no charset parameter
because json must always be encoded in one of utf-* for transmission).
Conversely, a byte array received over HTTP can, MIME type allowing,
decoded into a string.

The fact that there exist strings for which encoding and decoding are
identity transforms should be regarded only as an implementation
detail. Attempts by libraries and frameworks to silently DTRT for this
subset lead to applications neglecting to properly encode or tag
strings, leading, in turn, to breakage in presence of multilingual
text.