Map certs with ITUT X509 to our RSA implementation

[samuel40791765]

Description of changes:

There exists a rarer OID of RSA public keys which comes from the ITU-T version of X.509 (links below).
Links:

• https://www.itu.int/ITU-T/formal-language/itu-t/x/x509/2008/AlgorithmObjectIdentifiers.html
• https://www.ivi.co/jsse/2009/06/13/0220.html

OpenSSL accepts the sign_nid and pkey_nid for this draft.

• NID_md5WithRSA
• NID_sha1WithRSA
• NID_rsa

We can parse both sign_nids correctly. But we lack the logic to interpret NID_rsa in a certificate or map the parsed sign_nids to any RSA implementation.
OpenSSL maps the corresponding nids from the same draft, which means NID_md5WithRSA and NID_sha1WithRSA are mapped to NID_rsa and an additional EVP_PKEY_RSA2 is used to represent NID_rsa. The additional EVP_PKEY type introduces additional complexities however and the underlying functionality in EVP_PKEY_RSA2 is identical to it's more common EVP_PKEY_RSA counterpart.

We've been burned by multiple OpenSSL cert parsing differences in the past, so we should have logic to understand certs that have NID_rsa. We don't want to duplicate all the function pointers for EVP_PKEY_RSA2 however. Since the underlying RSA functions pointers are the same, I think it should be OK for us to

1. Map NID_md5WithRSA and NID_sha1WithRSA to our RSA implementation (NID_rsa_encryption).
2. Map NID_rsa to EVP_PKEY_RSA when parsing certificates.

Unfortunately, there's some stricter parsing in rsa_pub_decode that I've had to remove since it doesn't apply to both RFCs for RSA public keys. This is aligning with OpenSSL, which happens to ignore both of the early parameters specified in either.

• Documented in f6094e0
• OpenSSL code (which ignores the parameter): https://github.com/openssl/openssl/blob/8e0d479b98357bb20ab1bd073cf75f7d42531553/crypto/rsa/rsa_ameth.c#L76-L90

Call-outs:

Although we have EVP_PKEY_RSA2 and we could map the logic for that to EVP_PKEY_RSA in EVP_PKEY_type, I've intentionally left that out for now. We still want to discourage people from using this form, we just need to understand how to parse certificates that have it.

Relevant historic commits:

• Removed support for NID_rsa: f6094e0
• Sigalgs mapping removed: 720ff53
• RSA stricter parsing introduced: 68772b3

Testing:

WIP

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license and the ISC license.

[maddeleine]

I tested this change and can confirm that this addition to the kTriples table resolves this s2n-tls issue: aws/s2n-tls#4691

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 78.55%. Comparing base (c664abe) to head (9a12688).

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1754      +/-   ##
==========================================
- Coverage   78.56%   78.55%   -0.01%     
==========================================
  Files         583      583              
  Lines       98807    98808       +1     
  Branches    14161    14160       -1     
==========================================
- Hits        77623    77621       -2     
- Misses      20558    20559       +1     
- Partials      626      628       +2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.