Skip to content

Commit 6bb85ba

Browse files
kyessenovkyessenov
authored
certificate selector: on demand SDS (#42072)
Commit Message: add an extension to load certs via SDS on-demand instead of warming SDS secret Additional Description: the following objects are involved: - `AsyncSelector` is the handshake override that pauses the handshake with a callback; - `SecretManager` maintains the set of TLS context configurations and realized SSL objects; - `AsyncContextConfig` maintains an active SDS subscription and emits update events to `SecretManager` on SDS updates; - `AsyncContext` converts SDS certificate into SSL objects using the outer TLS configuration context; - `SelectionHandle` a callback registration into `SecretManager` from `AsyncSelector`. A basic flow is as follows: - `AsyncSelector` is invoked when a TLS handshake happens on the worker; - It queries `SecretManager` thread-local storage for an active `AsyncContext` for a mapped secret name; - If it finds one, it latches onto it, and supplies its inner SSL objects; - If it doesn't find one, it forwards the handshake callback to `SecretManager` on the main thread; - `SecretManager` responds by creating an active SDS subscription. It's possible a thread local did not make it to the worker, so it can immediately answer the callback. More likely, it needs to save the callback into a queue. - When a `SecretManager` receives an SDS response, it tries creating `AsyncContext` from it. If it succeeds, it can drain the callback queue and post a thread local update. - `SelectionHandle` is held as a weak_ptr to detect interrupted handshakes and avoid responding to them. Note that there is no locking involved, and at the steady state, certificates are available on all workers. Risk Level: low, new extension, secret code refactor should not impact current behavior Testing: added integration tests Docs Changes: yes Release Notes: yes Issue: #30600 --------- Signed-off-by: Kuat Yessenov <kuat@google.com>
1 parent 2c20937 commit 6bb85ba

96 files changed

Lines changed: 2564 additions & 776 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎CODEOWNERS‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -57,6 +57,10 @@ extensions/filters/common/original_src @klarose @mattklein123
5757
/*/extensions/transport_sockets/tls @RyanTheOptimist @ggreenway @botengyao
5858
# tls SPIFFE certificate validator extension
5959
/*/extensions/transport_sockets/tls/cert_validator/spiffe @mathetake @botengyao @tyxia
60+
# On demand secret provider
61+
/*/extensions/transport_sockets/tls/cert_selectors/on_demand @kyessenov @tonya11en
62+
/*/extensions/transport_sockets/tls/cert_mappers/static_name @kyessenov @tonya11en
63+
/*/extensions/transport_sockets/tls/cert_mappers/sni @kyessenov @tonya11en
6064
# proxy protocol socket extension
6165
/*/extensions/transport_sockets/proxy_protocol @botengyao @wez470
6266
# common transport socket

‎api/BUILD‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -386,6 +386,9 @@ proto_library(
386386
"//envoy/extensions/transport_sockets/starttls/v3:pkg",
387387
"//envoy/extensions/transport_sockets/tap/v3:pkg",
388388
"//envoy/extensions/transport_sockets/tcp_stats/v3:pkg",
389+
"//envoy/extensions/transport_sockets/tls/cert_mappers/sni/v3:pkg",
390+
"//envoy/extensions/transport_sockets/tls/cert_mappers/static_name/v3:pkg",
391+
"//envoy/extensions/transport_sockets/tls/cert_selectors/on_demand_secret/v3:pkg",
389392
"//envoy/extensions/transport_sockets/tls/v3:pkg",
390393
"//envoy/extensions/udp_packet_writer/v3:pkg",
391394
"//envoy/extensions/upstreams/http/generic/v3:pkg",

‎api/envoy/extensions/transport_sockets/tls/cert_mappers/sni/v3/BUILD‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
# DO NOT EDIT. This file is generated by tools/proto_format/proto_sync.py.
2+
3+
load("@envoy_api//bazel:api_build_system.bzl", "api_proto_package")
4+
5+
licenses(["notice"]) # Apache 2
6+
7+
api_proto_package(
8+
deps = ["@com_github_cncf_xds//udpa/annotations:pkg"],
9+
)

‎api/envoy/extensions/transport_sockets/tls/cert_mappers/sni/v3/config.proto‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
syntax = "proto3";
2+
3+
package envoy.extensions.transport_sockets.tls.cert_mappers.sni.v3;
4+
5+
import "udpa/annotations/status.proto";
6+
import "validate/validate.proto";
7+
8+
option java_package = "io.envoyproxy.envoy.extensions.transport_sockets.tls.cert_mappers.sni.v3";
9+
option java_outer_classname = "ConfigProto";
10+
option java_multiple_files = true;
11+
option go_package = "github.com/envoyproxy/go-control-plane/envoy/extensions/transport_sockets/tls/cert_mappers/sni/v3;sniv3";
12+
option (udpa.annotations.file_status).package_version_status = ACTIVE;
13+
14+
// [#protodoc-title: SNI certificate mapper]
15+
// [#extension: envoy.tls.certificate_mappers.sni]
16+
17+
// Uses the SNI value from the TLS client hello as the secret resource name.
18+
message SNI {
19+
// The value to use as the secret name when SNI is empty or absent.
20+
string default_value = 1 [(validate.rules).string = {min_len: 1}];
21+
}

‎api/envoy/extensions/transport_sockets/tls/cert_mappers/static_name/v3/BUILD‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
# DO NOT EDIT. This file is generated by tools/proto_format/proto_sync.py.
2+
3+
load("@envoy_api//bazel:api_build_system.bzl", "api_proto_package")
4+
5+
licenses(["notice"]) # Apache 2
6+
7+
api_proto_package(
8+
deps = ["@com_github_cncf_xds//udpa/annotations:pkg"],
9+
)

‎api/envoy/extensions/transport_sockets/tls/cert_mappers/static_name/v3/config.proto‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
syntax = "proto3";
2+
3+
package envoy.extensions.transport_sockets.tls.cert_mappers.static_name.v3;
4+
5+
import "udpa/annotations/status.proto";
6+
import "validate/validate.proto";
7+
8+
option java_package = "io.envoyproxy.envoy.extensions.transport_sockets.tls.cert_mappers.static_name.v3";
9+
option java_outer_classname = "ConfigProto";
10+
option java_multiple_files = true;
11+
option go_package = "github.com/envoyproxy/go-control-plane/envoy/extensions/transport_sockets/tls/cert_mappers/static_name/v3;static_namev3";
12+
option (udpa.annotations.file_status).package_version_status = ACTIVE;
13+
14+
// [#protodoc-title: Static secret certificate mapper]
15+
// [#extension: envoy.tls.certificate_mappers.static_name]
16+
17+
// A mapping to a fixed secret name for all certificates.
18+
message StaticName {
19+
// The name for the secret to use for all connections.
20+
string name = 1 [(validate.rules).string = {min_len: 1}];
21+
}

‎api/envoy/extensions/transport_sockets/tls/cert_selectors/on_demand_secret/v3/BUILD‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
# DO NOT EDIT. This file is generated by tools/proto_format/proto_sync.py.
2+
3+
load("@envoy_api//bazel:api_build_system.bzl", "api_proto_package")
4+
5+
licenses(["notice"]) # Apache 2
6+
7+
api_proto_package(
8+
deps = [
9+
"//envoy/config/core/v3:pkg",
10+
"@com_github_cncf_xds//udpa/annotations:pkg",
11+
],
12+
)

‎api/envoy/extensions/transport_sockets/tls/cert_selectors/on_demand_secret/v3/config.proto‎

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
syntax = "proto3";
2+
3+
package envoy.extensions.transport_sockets.tls.cert_selectors.on_demand_secret.v3;
4+
5+
import "envoy/config/core/v3/config_source.proto";
6+
import "envoy/config/core/v3/extension.proto";
7+
8+
import "udpa/annotations/status.proto";
9+
import "validate/validate.proto";
10+
11+
option java_package = "io.envoyproxy.envoy.extensions.transport_sockets.tls.cert_selectors.on_demand_secret.v3";
12+
option java_outer_classname = "ConfigProto";
13+
option java_multiple_files = true;
14+
option go_package = "github.com/envoyproxy/go-control-plane/envoy/extensions/transport_sockets/tls/cert_selectors/on_demand_secret/v3;on_demand_secretv3";
15+
option (udpa.annotations.file_status).package_version_status = ACTIVE;
16+
17+
// [#protodoc-title: On-demand secret certificate selector]
18+
// [#extension: envoy.tls.certificate_selectors.on_demand_secret]
19+
20+
// Fetches the secret on-demand while allowing the parent cluster or listener to accept connections
21+
// without warming. During the handshake, a secret name is derived from the peer hello message, an
22+
// SDS resource request starts, and the handshake is paused. Once an SDS response is received with a
23+
// resource, the handshake is resumed with the provided certificate. If the SDS server indicates the
24+
// resource removal, the handshake is failed, and the SDS subscription to the resource is stopped.
25+
//
26+
// Similar to the regular SDS, the certificate is configured using the outer common TLS context,
27+
// e.g. by setting the FIPS compliance policy on the loaded certificate.
28+
message Config {
29+
// Defines the configuration source of the secrets.
30+
config.core.v3.ConfigSource config_source = 1 [(validate.rules).message = {required: true}];
31+
32+
// Extension point to specify a function to compute the secret name. The extension is called
33+
// during the TLS handshake after receiving the "CLIENT HELLO" message from the client.
34+
// [#extension-category: envoy.tls.certificate_mappers]
35+
config.core.v3.TypedExtensionConfig certificate_mapper = 2
36+
[(validate.rules).message = {required: true}];
37+
38+
// A list of secret resource names to start fetching on configuration load (prior to receiving any
39+
// requests). The parent resource initializes immediately without waiting for the fetch to
40+
// complete.
41+
repeated string prefetch_secret_names = 3;
42+
}

‎api/envoy/extensions/transport_sockets/tls/v3/tls.proto‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -300,6 +300,7 @@ message CommonTlsContext {
300300
// Select TLS certificate based on TLS client hello.
301301
// If empty, defaults to native TLS certificate selection behavior:
302302
// DNS SANs or Subject Common Name in TLS certificates is extracted as server name pattern to match SNI.
303+
// [#extension-category: envoy.tls.certificate_selectors]
303304
config.core.v3.TypedExtensionConfig custom_tls_certificate_selector = 16;
304305

305306
// Certificate provider for fetching TLS certificates.

‎api/versioning/BUILD‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -326,6 +326,9 @@ proto_library(
326326
"//envoy/extensions/transport_sockets/starttls/v3:pkg",
327327
"//envoy/extensions/transport_sockets/tap/v3:pkg",
328328
"//envoy/extensions/transport_sockets/tcp_stats/v3:pkg",
329+
"//envoy/extensions/transport_sockets/tls/cert_mappers/sni/v3:pkg",
330+
"//envoy/extensions/transport_sockets/tls/cert_mappers/static_name/v3:pkg",
331+
"//envoy/extensions/transport_sockets/tls/cert_selectors/on_demand_secret/v3:pkg",
329332
"//envoy/extensions/transport_sockets/tls/v3:pkg",
330333
"//envoy/extensions/udp_packet_writer/v3:pkg",
331334
"//envoy/extensions/upstreams/http/generic/v3:pkg",

0 commit comments

Comments
 (0)