bug#10904: 24.0.93; Infinite loop in GnuTLS code during Gnus nnimap-initiated SSL handshake

[Thomas Fitzsimmons]

Ted Zlatanov <tzz@lifelogs.com> writes:

> On Sat, 03 Mar 2012 15:56:59 +0100 Lars Magne Ingebrigtsen <larsi@gnus.org> 
> wrote:
>
> LMI> Thomas Fitzsimmons <fitzsim@fitzsim.org> writes:
> LMI> The error seems to be this?
>
>>> gnutls.c: [4] REC[0x90155c8]: Decrypted Packet[1] Handshake(22) with
>>> length: 3404
>>>
>>> gnutls.c: [3] HSK[0x90155c8]: CERTIFICATE was received. Length
>>> 3400[3400], frag offset 0, frag length: 3400, sequence: 0
>>>
>>> gnutls.c: [2] ASSERT:
>>> /home/fitzsim/sources/gnutls-3.0.8/lib/gnutls_buffers.c:1037
>>>
>>> gnutls.c: [1] Received unexpected handshake message 'CERTIFICATE'
>>> (11). Expected 'SERVER HELLO' (2)
>>>
>>> gnutls.c: [2] ASSERT:
>>> /home/fitzsim/sources/gnutls-3.0.8/lib/gnutls_handshake.c:1226
>>>
>>> gnutls.c: [2] ASSERT:
>>> /home/fitzsim/sources/gnutls-3.0.8/lib/gnutls_handshake.c:2432
>>>
>>> gnutls.c: [0] (Emacs) fatal error: An unexpected TLS handshake packet
>>> was received.
>
> LMI> That is, GnuTLS expects SERVER HELLO, but gets CERTIFICATE from the IMAP
> LMI> server.
>
> LMI> Does anybody know what this might mean?
>
> (sorry for the late reply)
>
> It seems like a buggy server or something in GnuTLS itself.  I have
> never seen this problem before and the lockup is happening in the GnuTLS
> internals, where Emacs doesn't play at all (we use the GnuTLS API in a
> very straightforward way).
>
> Thomas, can you open a connection to this server with the GnuTLS
> command-line tools (gnutls-cli)?  That will reduce the problem to Emacs
> vs. GnuTLS.

I did some more investigation using the command line tools.  It turned
out that the IMAP server hostname was resolving to one of a set of
Microsoft Exchange front-end servers.  GnuTLS would fail to connect to
some of the front-end servers but not to others.  This explained why I
would only see the issue sometimes (about half the time).

By explicitly pointing <imap_hostname> in /etc/hosts at one or another
front-end server I was able to make the GnuTLS connection either pass or
fail consistently.  Then I tried "openssl s_client" and it worked on
both GnuTLS-passing and GnuTLS-failing servers.  The servers Gnus failed
to access reported:

[...]
---
New, TLSv1/SSLv3, Cipher is DES-CBC3-SHA
Server public key is 2048 bit
Compression: NONE
Expansion: NONE
SSL-Session:
    Protocol  : TLSv1
    Cipher    : DES-CBC3-SHA
    Session-ID: 56EFB2D45A0A7A15F173CE086AC0A754016BA00300602F30B47273E565792400
    Session-ID-ctx:
    Master-Key: 
CD97CB5CB685B42815C8FB056032ABF03C890B42790A07B570E6ED93AEC70D81970EE344265C133100D84BB3789E6B5D
    Key-Arg   : None
    Krb5 Principal: None
    Start Time: 1332336762
    Timeout   : 300 (sec)
    Verify return code: 0 (ok)
---
* OK [...]

and the ones it could access reported:

[...]
---
New, TLSv1/SSLv3, Cipher is RC4-MD5
Server public key is 2048 bit
Compression: NONE
Expansion: NONE
SSL-Session:
    Protocol  : TLSv1
    Cipher    : RC4-MD5
    Session-ID:
    Session-ID-ctx:
    Master-Key: 
B6A8BCC0224B72A00A34435DEA66580BD5BFA0FA5D1467471E8070968F206134B410715AE8808C754000B9F1BC1BBD84
    Key-Arg   : None
    Krb5 Principal: None
    Start Time: 1332271498
    Timeout   : 300 (sec)
    Verify return code: 0 (ok)
---
* OK [...]

All of the front-end Exchange servers reported the same version string
but some (the failing set) were configured with the DES-CBC3-SHA cipher
and others (the passing set) were configured with the RC4-MD5 cipher.

Then I went back to gnutls-cli to experiment with its supported ciphers
list.  I discovered that adding the --priority "PERFORMANCE" option
resulted in success for previously-passing and previously-failing server
connections.

The result from gnutls-cli for the previously-passing server
configuration after I added --priority "PERFORMANCE" was:

- The hostname in the certificate matches '<imap_hostname>'.
- Peer's certificate issuer is unknown
- Peer's certificate is NOT trusted
- Version: TLS1.0
- Key Exchange: RSA
- Cipher: ARCFOUR-128
- MAC: MD5
- Compression: NULL
- Session ID: (null)
- Channel binding 'tls-unique': c38c134cd89ef6f0ba9c51f9
- Handshake was completed

- Simple Client Mode:

[...]

- Received[104]: * OK [...]

which was the same as without the --priority option.  However the result
for the previously-failing server configuration after I added --priority
"PERFORMANCE" was:

- The hostname in the certificate matches '<imap_hostname>'.
- Peer's certificate issuer is unknown
- Peer's certificate is NOT trusted
- Version: TLS1.0
- Key Exchange: RSA
- Cipher: ARCFOUR-128
- MAC: SHA1
- Compression: NULL
- Session ID: 
35:B3:B2:CE:52:FF:00:D3:0F:DF:0C:25:E0:27:A8:CF:02:9D:17:03:00:14:1C:5B:F3:0F:46:86:02:69:49:64
- Channel binding 'tls-unique': 63b922f2b761712f1f25c40e
- Handshake was completed

- Simple Client Mode:

[...]

- Received[109]: * OK [...]

so --priority "PERFORMANCE" worked around the handshake failure by
selecting a cipher other than the failing DES-CBC3-SHA.

In Emacs I worked around the issue by customizing
gnutls-algorithm-priority to "performance".  Now Gnus always connects
successfully.

But there are still two issues:

1) GnuTLS fails to handshake with a server that uses the DES-CBC3-SHA
   cipher, whereas OpenSSL succeeds

2) If gnutls.el fails to handshake with a server then Emacs enters an
   infinite loop retrying the handshake

Thomas