One leading NUL byte in RSA key modulus

[Jim Lloyd]

Hello,

I am nearly done porting our application to use gnutls 2.12 instead of 2.8. We have a reasonably decent set of unit tests, nearly all of which pass without requiring any changes. However, there are two failing tests that I find curious.

First, we have a function that user gnutls_x509_privkey_export_rsa_raw to return the size in bytes of the modulus of a private key. One specific unit test simply asserts that the size of a given test key is 128 bytes. That test fails because the actual size of the key is now 129 bytes. 

We have another test that uses gnutls_x509_crt_print to print a human readable description of a certificate. The test simply verifies the output is identical to an expected string. That test fails because the modulus now shows an additional leading 00 byte. For example:

                Modulus (bits 1024):

                        00:b5:b8:d2:04:ef:8b:ed:de:91:14:4e:6c:f5:70:da

                        22:f1:30:ca:48:f9:7f:b2:eb:4f:d5:cc:8d:ac:cc:37

                        76:ca:21:22:b1:c0:1c:74:c4:39:32:08:18:43:04:cf

                        92:19:72:9e:fb:a2:65:c3:97:56:81:a3:31:73:33:c9

                        a3:57:8d:a1:9e:6d:91:97:a9:db:e8:5c:e8:2f:c0:71

                        db:54:4e:01:79:e8:a7:9b:03:70:8e:db:21:8e:21:9f

                        a9:5b:08:19:40:b2:bb:58:e5:09:94:7e:94:65:33:6d

                        e7:f0:63:db:b2:76:a7:55:8f:2e:28:c6:e6:c6:e9:c5

                        85

In addition, the Public Key Id output by gnutls_x509_crt_print has changed to be an entirely different hash.

If I change the expected results in these two unit tests to reflect these changes, then my all of our unit tests completely pass. Still, it seems odd to me that this behavior has changed. Is the change completely benign? What was the reason for the change?

Thanks,

Jim Lloyd