Add functions for manipulating X.509 TBS structures.

When generating a signature with some external signing process, the
caller needs to fill in the TBSCertificate (including the signature
algorithms), serialize the TBSCertificate, and then fill in the
signature.

We have i2d_re_X509_tbs (originally from CT I believe), but there are no
setters for the signature algorithms or the signature. Add
X509_set1_signature_algo, which mirrors upstream's
X509_REQ_set1_signature_algo, and X509_set1_signature_value, which is
new. Upstream has X509_REQ_set0_signature, but that requires the caller
manually assemble an ASN1_BIT_STRING. Taking the byte string seems less
error-prone.

Additionally, add i2d_X509_tbs and i2d_X509_CRL_tbs, for the non-"re"
variants of those APIs. Conscrypt needs to extract the TBS portion of a
certificate and a CRL, to implement X509Certificate.getTBSCertificate()
and X509CRL.getTBSCertList(). There, the aim is to get the data to
verify on an existing immutable certificate. OpenSSL has avoided
exporting the X509_CINF type, which I think is correct, so instead this
mirrors i2d_re_X509_tbs. (This does mean mirroring the confusing i2d
calling convention though.)

These new functions should unblock getting rid of a bunch of direct
struct accesses.

Later on, we should reorganize this header into immutable APIs for
verification and mutable APIs for generation. Even though we're stuck
the mistake of a common type for both use cases, I think splitting up
them up will let us rationalize the caches in the X509 objects a bit.

Change-Id: I96e6ab5cee3608e07b2ed7465c449a72ca10a393
Reviewed-on: https://boringssl-review.googlesource.com/c/boringssl/+/43784
Commit-Queue: Adam Langley <agl@google.com>
Reviewed-by: Adam Langley <agl@google.com>
diff --git a/crypto/x509/x509_test.cc b/crypto/x509/x509_test.cc
index b170b76..7c0eeac 100644
--- a/crypto/x509/x509_test.cc
+++ b/crypto/x509/x509_test.cc
@@ -1510,6 +1510,18 @@
             Bytes(public_value, public_value_size));
 }
 
+static bssl::UniquePtr<X509> ReencodeCertificate(X509 *cert) {
+  uint8_t *der = nullptr;
+  int len = i2d_X509(cert, &der);
+  bssl::UniquePtr<uint8_t> free_der(der);
+  if (len <= 0) {
+    return nullptr;
+  }
+
+  const uint8_t *inp = der;
+  return bssl::UniquePtr<X509>(d2i_X509(nullptr, &inp, len));
+}
+
 static bool SignatureRoundTrips(EVP_MD_CTX *md_ctx, EVP_PKEY *pkey) {
   // Make a certificate like signed with |md_ctx|'s settings.'
   bssl::UniquePtr<X509> cert(CertFromPEM(kLeafPEM));
@@ -1519,7 +1531,14 @@
 
   // Ensure that |pkey| may still be used to verify the resulting signature. All
   // settings in |md_ctx| must have been serialized appropriately.
-  return !!X509_verify(cert.get(), pkey);
+  if (!X509_verify(cert.get(), pkey)) {
+    return false;
+  }
+
+  // Re-encode the certificate. X509 objects contain a cached TBSCertificate
+  // encoding and |X509_sign_ctx| should have refreshed that cache.
+  bssl::UniquePtr<X509> copy = ReencodeCertificate(cert.get());
+  return copy && X509_verify(copy.get(), pkey);
 }
 
 TEST(X509Test, RSASign) {
@@ -1541,6 +1560,83 @@
   ASSERT_TRUE(SignatureRoundTrips(md_ctx.get(), pkey.get()));
 }
 
+// Test the APIs for manually signing a certificate.
+TEST(X509Test, RSASignManual) {
+  const int kSignatureNID = NID_sha384WithRSAEncryption;
+  const EVP_MD *kSignatureHash = EVP_sha384();
+
+  bssl::UniquePtr<EVP_PKEY> pkey(PrivateKeyFromPEM(kRSAKey));
+  ASSERT_TRUE(pkey);
+  bssl::UniquePtr<X509_ALGOR> algor(X509_ALGOR_new());
+  ASSERT_TRUE(algor);
+  ASSERT_TRUE(X509_ALGOR_set0(algor.get(), OBJ_nid2obj(kSignatureNID),
+                              V_ASN1_NULL, nullptr));
+
+  // Test certificates made both from other certificates and |X509_new|, in case
+  // there are bugs in filling in fields from different states. (Parsed
+  // certificate contain a TBSCertificate cache, and |X509_new| initializes
+  // fields based on complex ASN.1 template logic.)
+  for (bool new_cert : {true, false}) {
+    SCOPED_TRACE(new_cert);
+
+    bssl::UniquePtr<X509> cert;
+    if (new_cert) {
+      cert.reset(X509_new());
+      // Fill in some fields for the certificate arbitrarily.
+      EXPECT_TRUE(X509_set_version(cert.get(), 2 /* X.509v3 */));
+      EXPECT_TRUE(ASN1_INTEGER_set(X509_get_serialNumber(cert.get()), 1));
+      EXPECT_TRUE(X509_gmtime_adj(X509_getm_notBefore(cert.get()), 0));
+      EXPECT_TRUE(
+          X509_gmtime_adj(X509_getm_notAfter(cert.get()), 60 * 60 * 24));
+      X509_NAME *subject = X509_get_subject_name(cert.get());
+      X509_NAME_add_entry_by_txt(subject, "CN", MBSTRING_ASC,
+                                 reinterpret_cast<const uint8_t *>("Test"), -1,
+                                 -1, 0);
+      EXPECT_TRUE(X509_set_issuer_name(cert.get(), subject));
+      EXPECT_TRUE(X509_set_pubkey(cert.get(), pkey.get()));
+    } else {
+      // Extract fields from a parsed certificate.
+      cert = CertFromPEM(kLeafPEM);
+      ASSERT_TRUE(cert);
+
+      // We should test with a different algorithm from what is already in the
+      // certificate.
+      EXPECT_NE(kSignatureNID, X509_get_signature_nid(cert.get()));
+    }
+
+    // Fill in the signature algorithm.
+    ASSERT_TRUE(X509_set1_signature_algo(cert.get(), algor.get()));
+
+    // Extract the TBSCertificiate.
+    uint8_t *tbs_cert = nullptr;
+    int tbs_cert_len = i2d_re_X509_tbs(cert.get(), &tbs_cert);
+    bssl::UniquePtr<uint8_t> free_tbs_cert(tbs_cert);
+    ASSERT_GT(tbs_cert_len, 0);
+
+    // Generate a signature externally and fill it in.
+    bssl::ScopedEVP_MD_CTX md_ctx;
+    ASSERT_TRUE(EVP_DigestSignInit(md_ctx.get(), nullptr, kSignatureHash,
+                                   nullptr, pkey.get()));
+    size_t sig_len;
+    ASSERT_TRUE(EVP_DigestSign(md_ctx.get(), nullptr, &sig_len, tbs_cert,
+                               tbs_cert_len));
+    std::vector<uint8_t> sig(sig_len);
+    ASSERT_TRUE(EVP_DigestSign(md_ctx.get(), sig.data(), &sig_len, tbs_cert,
+                               tbs_cert_len));
+    sig.resize(sig_len);
+    ASSERT_TRUE(X509_set1_signature_value(cert.get(), sig.data(), sig.size()));
+
+    // Check the signature.
+    EXPECT_TRUE(X509_verify(cert.get(), pkey.get()));
+
+    // Re-encode the certificate. X509 objects contain a cached TBSCertificate
+    // encoding and |i2d_re_X509_tbs| should have refreshed that cache.
+    bssl::UniquePtr<X509> copy = ReencodeCertificate(cert.get());
+    ASSERT_TRUE(copy);
+    EXPECT_TRUE(X509_verify(copy.get(), pkey.get()));
+  }
+}
+
 TEST(X509Test, Ed25519Sign) {
   uint8_t pub_bytes[32], priv_bytes[64];
   ED25519_keypair(pub_bytes, priv_bytes);
diff --git a/crypto/x509/x509cset.c b/crypto/x509/x509cset.c
index b07ff27..a5cb3a3 100644
--- a/crypto/x509/x509cset.c
+++ b/crypto/x509/x509cset.c
@@ -239,8 +239,13 @@
     return r->extensions;
 }
 
-int i2d_re_X509_CRL_tbs(X509_CRL *crl, unsigned char **pp)
+int i2d_re_X509_CRL_tbs(X509_CRL *crl, unsigned char **outp)
 {
     crl->crl->enc.modified = 1;
-    return i2d_X509_CRL_INFO(crl->crl, pp);
+    return i2d_X509_CRL_INFO(crl->crl, outp);
+}
+
+int i2d_X509_CRL_tbs(X509_CRL *crl, unsigned char **outp)
+{
+    return i2d_X509_CRL_INFO(crl->crl, outp);
 }
diff --git a/crypto/x509/x_x509.c b/crypto/x509/x_x509.c
index ab24651..397230c 100644
--- a/crypto/x509/x_x509.c
+++ b/crypto/x509/x_x509.c
@@ -337,10 +337,45 @@
     return length;
 }
 
-int i2d_re_X509_tbs(X509 *x, unsigned char **pp)
+int i2d_re_X509_tbs(X509 *x509, unsigned char **outp)
 {
-    x->cert_info->enc.modified = 1;
-    return i2d_X509_CINF(x->cert_info, pp);
+    x509->cert_info->enc.modified = 1;
+    return i2d_X509_CINF(x509->cert_info, outp);
+}
+
+int i2d_X509_tbs(X509 *x509, unsigned char **outp)
+{
+    return i2d_X509_CINF(x509->cert_info, outp);
+}
+
+int X509_set1_signature_algo(X509 *x509, const X509_ALGOR *algo)
+{
+    /* TODO(davidben): Const-correct generated ASN.1 dup functions.
+     * Alternatively, when the types are hidden and we can embed required fields
+     * directly in structs, import |X509_ALGOR_copy| from upstream. */
+    X509_ALGOR *copy1 = X509_ALGOR_dup((X509_ALGOR *)algo);
+    X509_ALGOR *copy2 = X509_ALGOR_dup((X509_ALGOR *)algo);
+    if (copy1 == NULL || copy2 == NULL) {
+        X509_ALGOR_free(copy1);
+        X509_ALGOR_free(copy2);
+        return 0;
+    }
+
+    X509_ALGOR_free(x509->sig_alg);
+    x509->sig_alg = copy1;
+    X509_ALGOR_free(x509->cert_info->signature);
+    x509->cert_info->signature = copy2;
+    return 1;
+}
+
+int X509_set1_signature_value(X509 *x509, const uint8_t *sig, size_t sig_len)
+{
+    if (!ASN1_STRING_set(x509->signature, sig, sig_len)) {
+      return 0;
+    }
+    x509->signature->flags &= ~(ASN1_STRING_FLAG_BITS_LEFT | 0x07);
+    x509->signature->flags |= ASN1_STRING_FLAG_BITS_LEFT;
+    return 1;
 }
 
 void X509_get0_signature(const ASN1_BIT_STRING **psig, const X509_ALGOR **palg,
diff --git a/include/openssl/x509.h b/include/openssl/x509.h
index 38a4219..580c591 100644
--- a/include/openssl/x509.h
+++ b/include/openssl/x509.h
@@ -899,7 +899,51 @@
 OPENSSL_EXPORT X509 *d2i_X509_AUX(X509 **a, const unsigned char **pp,
                                   long length);
 
-OPENSSL_EXPORT int i2d_re_X509_tbs(X509 *x, unsigned char **pp);
+// i2d_re_X509_tbs serializes the TBSCertificate portion of |x509|. If |outp| is
+// NULL, nothing is written. Otherwise, if |*outp| is not NULL, the result is
+// written to |*outp|, which must have enough space available, and |*outp| is
+// advanced just past the output. If |outp| is non-NULL and |*outp| is NULL, it
+// sets |*outp| to a newly-allocated buffer containing the result. The caller is
+// responsible for releasing the buffer with |OPENSSL_free|. In all cases, this
+// function returns the number of bytes in the result, whether written or not,
+// or a negative value on error.
+//
+// This function re-encodes the TBSCertificate and may not reflect |x509|'s
+// original encoding. It may be used to manually generate a signature for a new
+// certificate. To verify certificates, use |i2d_X509_tbs| instead.
+OPENSSL_EXPORT int i2d_re_X509_tbs(X509 *x509, unsigned char **outp);
+
+// i2d_X509_tbs serializes the TBSCertificate portion of |x509|. If |outp| is
+// NULL, nothing is written. Otherwise, if |*outp| is not NULL, the result is
+// written to |*outp|, which must have enough space available, and |*outp| is
+// advanced just past the output. If |outp| is non-NULL and |*outp| is NULL, it
+// sets |*outp| to a newly-allocated buffer containing the result. The caller is
+// responsible for releasing the buffer with |OPENSSL_free|. In all cases, this
+// function returns the number of bytes in the result, whether written or not,
+// or a negative value on error.
+//
+// This function preserves the original encoding of the TBSCertificate and may
+// not reflect modifications made to |x509|. It may be used to manually verify
+// the signature of an existing certificate. To generate certificates, use
+// |i2d_re_X509_tbs| instead.
+OPENSSL_EXPORT int i2d_X509_tbs(X509 *x509, unsigned char **outp);
+
+// X509_set1_signature_algo sets |x509|'s signature algorithm to |algo| and
+// returns one on success or zero on error. It updates both the signature field
+// of the TBSCertificate structure, and the signatureAlgorithm field of the
+// Certificate.
+OPENSSL_EXPORT int X509_set1_signature_algo(X509 *x509, const X509_ALGOR *algo);
+
+// X509_set1_signature_value sets |x509|'s signature to a copy of the |sig_len|
+// bytes pointed by |sig|. It returns one on success and zero on error.
+//
+// Due to a specification error, X.509 certificates store signatures in ASN.1
+// BIT STRINGs, but signature algorithms return byte strings rather than bit
+// strings. This function creates a BIT STRING containing a whole number of
+// bytes, with the bit order matching the DER encoding. This matches the
+// encoding used by all X.509 signature algorithms.
+OPENSSL_EXPORT int X509_set1_signature_value(X509 *x509, const uint8_t *sig,
+                                             size_t sig_len);
 
 OPENSSL_EXPORT void X509_get0_signature(const ASN1_BIT_STRING **psig,
                                         const X509_ALGOR **palg, const X509 *x);
@@ -1020,7 +1064,35 @@
                                             const ASN1_BIT_STRING **psig,
                                             const X509_ALGOR **palg);
 OPENSSL_EXPORT int X509_CRL_get_signature_nid(const X509_CRL *crl);
-OPENSSL_EXPORT int i2d_re_X509_CRL_tbs(X509_CRL *req, unsigned char **pp);
+
+// i2d_re_X509_CRL_tbs serializes the TBSCertList portion of |crl|. If |outp| is
+// NULL, nothing is written. Otherwise, if |*outp| is not NULL, the result is
+// written to |*outp|, which must have enough space available, and |*outp| is
+// advanced just past the output. If |outp| is non-NULL and |*outp| is NULL, it
+// sets |*outp| to a newly-allocated buffer containing the result. The caller is
+// responsible for releasing the buffer with |OPENSSL_free|. In all cases, this
+// function returns the number of bytes in the result, whether written or not,
+// or a negative value on error.
+//
+// This function re-encodes the TBSCertList and may not reflect |crl|'s original
+// encoding. It may be used to manually generate a signature for a new CRL. To
+// verify CRLs, use |i2d_X509_CRL_tbs| instead.
+OPENSSL_EXPORT int i2d_re_X509_CRL_tbs(X509_CRL *crl, unsigned char **outp);
+
+// i2d_X509_CRL_tbs serializes the TBSCertList portion of |crl|. If |outp| is
+// NULL, nothing is written. Otherwise, if |*outp| is not NULL, the result is
+// written to |*outp|, which must have enough space available, and |*outp| is
+// advanced just past the output. If |outp| is non-NULL and |*outp| is NULL, it
+// sets |*outp| to a newly-allocated buffer containing the result. The caller is
+// responsible for releasing the buffer with |OPENSSL_free|. In all cases, this
+// function returns the number of bytes in the result, whether written or not,
+// or a negative value on error.
+//
+// This function preserves the original encoding of the TBSCertList and may not
+// reflect modifications made to |crl|. It may be used to manually verify the
+// signature of an existing CRL. To generate CRLs, use |i2d_re_X509_CRL_tbs|
+// instead.
+OPENSSL_EXPORT int i2d_X509_CRL_tbs(X509_CRL *crl, unsigned char **outp);
 
 OPENSSL_EXPORT const ASN1_INTEGER *X509_REVOKED_get0_serialNumber(
     const X509_REVOKED *x);