crypto/x509 MTC: Tighten MTC subjectPublicKeyInfoHash length check

When computing the MTC entry hash, we write the subjectPublicKeyInfoHash
OCTET STRING with a single short-form length byte. Lengths above 127
require a long-form DER length. No EVP_MD outputs more than 127 bytes,
so this is unreachable, but tighten the check rather than support the
long form.

Change-Id: I877df064197c90a14689ec4244be7ba66a6a6964
Reviewed-on: https://boringssl-review.googlesource.com/c/boringssl/+/103908
Reviewed-by: David Benjamin <davidben@google.com>
Auto-Submit: Lily Chen <chlily@google.com>
Commit-Queue: Lily Chen <chlily@google.com>

crypto/x509/x509_mtc.cc

diff --git a/crypto/x509/x509_mtc.cc b/crypto/x509/x509_mtc.cc
index d457e1b..abbbbb4 100644
--- a/crypto/x509/x509_mtc.cc
+++ b/crypto/x509/x509_mtc.cc
@@ -545,9 +545,13 @@
   }
   uint8_t spki_digest[EVP_MAX_MD_SIZE];
   unsigned int spki_digest_len;
+  // The short-form DER length can only encode lengths up to 127 (0x7f). The MTC
+  // spec permits longer hash lengths up to 255 (0xff), which would require a
+  // long-form length, but no `EVP_MD` currently outputs more than 127 bytes, so
+  // reject them here instead of supporting both short and long encoding forms.
   if (!EVP_Digest(CBS_data(&spki), CBS_len(&spki), spki_digest,
                   &spki_digest_len, issuer_mtc_ca.log_hash(), nullptr) ||
-      spki_digest_len == 0 || spki_digest_len > 0xff) {
+      spki_digest_len == 0 || spki_digest_len > 0x7f) {
     OPENSSL_PUT_ERROR(X509, ERR_R_INTERNAL_ERROR);
     return 0;
   }