Make SSL_select_next_proto more robust to invalid calls.

SSL_select_next_proto has some fairly complex preconditions:

- The peer and supported list must be valid protocol lists
- The supported list must not be empty. The peer list may be empty due
  to one of NPN's edge cases.

In the context of how this function is meant to be used, these are
reasonable preconditions. The caller should not serialize its own list
wrong, and it makes no sense to try to negotiate a protocol when you
don't support any protocols. In particular, it complicates NPN's weird
"opportunistic" protocol.

However, the preconditions are unchecked and a bit subtle. Violating
them will result in memory errors. Bad syntax on the protocol lists is
mostly not a concern (you should encode your own list correctly and the
library checks the peer's list), but the second rule is somewhat of a
mess in practice:

Despite having the same precondition in reality, OpenSSL did not
document this. Their documentation implies things which are impossible
without this precondition, but they forgot to actually write down the
precondition. There's an added complexity that OpenSSL never updated the
parameter names to match the role reversal between ALPN and NPN.

There are thus a few cases where a buggy caller may pass an empty
"supported" list.

- An NPN client called SSL_select_next_proto despite not actually
  supporting any NPN protocols.

- An NPN client called SSL_select_next_proto, flipped the parameters,
  and the server advertised no protocols.

- An ALPN server called SSL_select_next_proto, passed its own list in as
  the second parameter, despite not actually supporting any ALPN
  protocols.

In these scenarios, the "opportunistic" protocol returned in the
OPENSSL_NPN_NO_OVERLAP case will be out of bounds. If the caller
discards it, this does not matter. If the caller returns it through the
NPN or ALPN selection callback, they have a problem. ALPN servers are
expected to discard it, though some may be buggy. NPN clients may
implement either behavior.

Older versions of some callers have exhibited variations on the above
mistakes, so empirically folks don't always get it right. OpenSSL's
wrong documentation also does not help matters. Instead, have
SSL_select_next_proto just check these preconditions. That is not a
performance-sensitive function and these preconditions are easy to
check. While I'm here, rewrite it with CBS so it is much more
straightforwardly correct.

What to return when the preconditions fail is tricky, but we need to
output *some* protocol, so we output the empty protocol. This, per the
previous test and doc fixes, is actually fine in NPN, so one of the
above buggy callers is not retroactively made OK. But it is not fine in
ALPN, so we still need to document that callers need to avoid this
state. To that end, revamp the documentation a bit.

Thanks to Joe Birr-Pixton for reporting this!

Fixed: 735
Change-Id: I4378a082385e7334e6abaa6705e6b15d6843f6c5
Reviewed-on: https://boringssl-review.googlesource.com/c/boringssl/+/69028
Reviewed-by: Bob Beck <bbe@google.com>
Commit-Queue: David Benjamin <davidben@google.com>
5 files changed
tree: 0a3dab51852bfa5e04be4debc2bb7e4dc4652144
  1. .github/
  2. cmake/
  3. crypto/
  4. decrepit/
  5. fuzz/
  6. gen/
  7. include/
  8. pki/
  9. rust/
  10. ssl/
  11. third_party/
  12. tool/
  13. util/
  14. .bazelignore
  15. .bazelrc
  16. .clang-format
  17. .gitignore
  18. API-CONVENTIONS.md
  19. BREAKING-CHANGES.md
  20. BUILD.bazel
  21. build.json
  22. BUILDING.md
  23. CMakeLists.txt
  24. codereview.settings
  25. CONTRIBUTING.md
  26. FUZZING.md
  27. go.mod
  28. go.sum
  29. INCORPORATING.md
  30. LICENSE
  31. MODULE.bazel
  32. MODULE.bazel.lock
  33. PORTING.md
  34. PrivacyInfo.xcprivacy
  35. README.md
  36. SANDBOXING.md
  37. STYLE.md
README.md

BoringSSL

BoringSSL is a fork of OpenSSL that is designed to meet Google's needs.

Although BoringSSL is an open source project, it is not intended for general use, as OpenSSL is. We don't recommend that third parties depend upon it. Doing so is likely to be frustrating because there are no guarantees of API or ABI stability.

Programs ship their own copies of BoringSSL when they use it and we update everything as needed when deciding to make API changes. This allows us to mostly avoid compromises in the name of compatibility. It works for us, but it may not work for you.

BoringSSL arose because Google used OpenSSL for many years in various ways and, over time, built up a large number of patches that were maintained while tracking upstream OpenSSL. As Google's product portfolio became more complex, more copies of OpenSSL sprung up and the effort involved in maintaining all these patches in multiple places was growing steadily.

Currently BoringSSL is the SSL library in Chrome/Chromium, Android (but it's not part of the NDK) and a number of other apps/programs.

Project links:

There are other files in this directory which might be helpful: