Using a configuration provider to externalize secrets - Amazon Managed Streaming for Apache Kafka

Using a configuration provider to externalize secrets

To externalize sensitive configuration values with a service like AWS Secrets Manager, you can set up a configuration provider that implements the ConfigProvider class interface. A configuration provider lets you specify variables instead of plaintext in a connector or worker configuration, and workers running in your connector resolve these variables at runtime. This prevents credentials and other secrets from appearing as plaintext in a connector configuration.

To develop your own configuration provider plugin, use the following guidelines:

Considerations

Consider the following when you use a configuration provider with Amazon MSK Connect.

  • Sensitive configuration values can appear in connector logs if a plugin does not define those values as secret. Kafka Connect treats undefined configuration values the same as any other plaintext value. To learn more, see Preventing secrets from appearing in connector logs.

  • By default, MSK Connect frequently restarts a connector when the connector uses a configuration provider. To turn off this restart behavior, you can set the config.action.reload value to none in your connector configuration.