Bound the overall output size of ASN1_generate_v3

The output of ASN1_generate_v3 is *mostly* linear with the input, except
SEQ and SET reference config sections. Sections can be referenced
multiple times, and so the structure grows exponentially.

Cap the total output size to mitigate this. As before, we don't consider
these functions safe to use with untrusted inputs, but unbounded growth
will frustrate fuzzing. This CL sets the limit to 64K, which should be
enough for anyone. (This is the size of a single X.509 extension,
whereas certificates themselves should not get that large.)

While not strictly necessary, this also rearranges the
ASN1_mbstring_copy call to pass in a maximum output. This portion does
scale linearly with the output, so it's fine, but the fuzzer discovered
an input with a 700K-byte input, which, with fuzzer instrumentation and
sanitizers, seems to be a bit slow. This change should help the fuzzer
get past those cases faster.

Update-Note: The stringly-typed API for constructing X.509 extensions
now has a maximum output size. If anyone was constructing an extension
larger than 64K, this will break. This is unlikely and should be caught
by unit tests; if a project hits this outside of tests, that means they
are passing untrusted input into this function, which is a security
vulnerability in itself, and means they especially need this change to
avoid a DoS.

Bug: oss-fuzz:55725
Change-Id: Ibb65854293f44bf48ed5855016ef7cd46d2fae77
Reviewed-on: https://boringssl-review.googlesource.com/c/boringssl/+/57125
Reviewed-by: Bob Beck <bbe@google.com>
Commit-Queue: David Benjamin <davidben@google.com>
Auto-Submit: David Benjamin <davidben@google.com>

crypto/asn1/a_mbstr.c

diff --git a/crypto/asn1/a_mbstr.c b/crypto/asn1/a_mbstr.c
index 81916c2..e1a3df2 100644
--- a/crypto/asn1/a_mbstr.c
+++ b/crypto/asn1/a_mbstr.c
@@ -162,20 +162,16 @@
 
     nchar++;
     utf8_len += cbb_get_utf8_len(c);
+    if (maxsize > 0 && nchar > (size_t)maxsize) {
+      OPENSSL_PUT_ERROR(ASN1, ASN1_R_STRING_TOO_LONG);
+      ERR_add_error_dataf("maxsize=%ld", maxsize);
+      return -1;
+    }
   }
 
-  char strbuf[32];
   if (minsize > 0 && nchar < (size_t)minsize) {
     OPENSSL_PUT_ERROR(ASN1, ASN1_R_STRING_TOO_SHORT);
-    BIO_snprintf(strbuf, sizeof strbuf, "%ld", minsize);
-    ERR_add_error_data(2, "minsize=", strbuf);
-    return -1;
-  }
-
-  if (maxsize > 0 && nchar > (size_t)maxsize) {
-    OPENSSL_PUT_ERROR(ASN1, ASN1_R_STRING_TOO_LONG);
-    BIO_snprintf(strbuf, sizeof strbuf, "%ld", maxsize);
-    ERR_add_error_data(2, "maxsize=", strbuf);
+    ERR_add_error_dataf("minsize=%ld", minsize);
     return -1;
   }