tree 6a059a2214de31509ca1bbb62537cbf3b6eebcef
parent e106b536ee6233fec2e8876a16686d10607911c5
author Theo Buehler <theorbuehler@gmail.com> 1685699181 +0200
committer Boringssl LUCI CQ <boringssl-scoped@luci-project-accounts.iam.gserviceaccount.com> 1685729800 +0000

Make BN_mod_inverse() deal with repeated arguments

BN_nnmod() can deal with the situation that the first and the second
arguments are the same, but it cannot deal with the first and the
second argument being equal. In that situation, if BN_mod(x, y, x, ctx)
results in a negative x, then the result of BN_nnmod() is zero. This
breaks the strange BN_mod_inverse(m, a, m, ctx).

Reported by Guido Vranken in
https://github.com/openssl/openssl/issues/21110

Change-Id: I8584720660f214f172b3b33716a5e3b29e8f2fd8
Reviewed-on: https://boringssl-review.googlesource.com/c/boringssl/+/60365
Reviewed-by: Bob Beck <bbe@google.com>
Reviewed-by: David Benjamin <davidben@google.com>
Commit-Queue: David Benjamin <davidben@google.com>