tree cd631128059a5bf84ee7a1a3907a0d7db1472289
parent 30bcb3bd283ac992e7b1e8848fd450ad560b4122
author Adam Langley <agl@google.com> 1492645208 -0700
committer CQ bot account: commit-bot@chromium.org <commit-bot@chromium.org> 1492702853 +0000

Fix several issues with prime numbers.

Firstly, FIPS 186-4 C.3.2 is broken for w=3. In step 4.1 it generates a
random, 2-bit number but in step 4.2 it rejects all four possible values
and loops forever.

Secondly, BN_is_prime_fasttext_ex is broken when trial division is
requested and the prime is small. It finds that the prime is a multiple
of a known prime and rejects it. We inherited this from OpenSSL.

Thirdly, we were missing a BN_CTX_start/end in
BN_enhanced_miller_rabin_primality_test, which didn't matter but could
have mattered in the future.

Change-Id: Ie988e37b14bb22acb005fc0652860be6bbd2a55f
Reviewed-on: https://boringssl-review.googlesource.com/15264
Reviewed-by: Adam Langley <agl@google.com>
Commit-Queue: Adam Langley <agl@google.com>
CQ-Verified: CQ bot account: commit-bot@chromium.org <commit-bot@chromium.org>