aws-dynamodb-stream-lambda-elasticsearch-kibana - AWS Solutions Constructs

aws-dynamodb-stream-lambda-elasticsearch-kibana

All classes are under active development and subject to non-backward compatible changes or removal in any future version. These are not subject to the Semantic Versioning model. This means that while you may use them, you may need to update your source code when upgrading to a newer version of this package.

Language Package
Python
aws_solutions_constructs.aws_dynamodb_stream_lambda_elasticsearch_kibana
Typescript
@aws-solutions-constructs/aws-dynamodb-stream-lambda-elasticsearch-kibana
Java
software.amazon.awsconstructs.services.dynamodbstreamlambdaelasticsearchkibana

This AWS Solutions Construct implements Amazon DynamoDB table with stream, an AWS Lambda function, and an Amazon Elasticsearch Service with the least privileged permissions.

Here is a minimal deployable pattern definition in TypeScript:

import { DynamoDBStreamToLambdaToElasticSearchAndKibana, DynamoDBStreamToLambdaToElasticSearchAndKibanaProps } from '@aws-solutions-constructs/aws-dynamodb-stream-lambda-elasticsearch-kibana'; import { Aws } from "@aws-cdk/core"; const props: DynamoDBStreamToLambdaToElasticSearchAndKibanaProps = { lambdaFunctionProps: { code: lambda.Code.fromAsset(`${__dirname}/lambda`), runtime: lambda.Runtime.NODEJS_12_X, handler: 'index.handler' }, domainName: 'test-domain', // TODO: Ensure the Cognito domain name is globally unique cognitoDomainName: 'globallyuniquedomain' + Aws.ACCOUNT_ID; }; new DynamoDBStreamToLambdaToElasticSearchAndKibana(this, 'test-dynamodb-stream-lambda-elasticsearch-kibana', props);

Initializer

new DynamoDBStreamToLambdaToElasticSearchAndKibana(scope: Construct, id: string, props: DynamoDBStreamToLambdaToElasticSearchAndKibanaProps);

Parameters

Pattern Construct Props

Name Type Description
existingLambdaObj? lambda.Function An optional, existing Lambda function to be used instead of the default function. If an existing function is provided, the lambdaFunctionProps property will be ignored.
lambdaFunctionProps? lambda.FunctionProps Optional user-provided properties to override the default properties for the Lambda function. Ignored if an existingLambdaObj is provided.
dynamoTableProps? dynamodb.TableProps Optional user provided props to override the default props for DynamoDB Table
existingTableObj? dynamodb.Table Existing instance of DynamoDB Table. If this is set, then the dynamoTableProps is ignored.
dynamoEventSourceProps? aws-lambda-event-sources.DynamoEventSourceProps Optional user provided props to override the default props for DynamoDB Event Source
esDomainProps? elasticsearch.CfnDomainProps Optional user provided props to override the default props for the Amazon Elasticsearch Service
domainName string Domain name for the Cognito and the Amazon Elasticsearch Service
createCloudWatchAlarms boolean Whether to create recommended CloudWatch alarms.

Pattern Properties

Name Type Description
cloudwatchAlarms? cloudwatch.Alarm[] Returns a list of one or more CloudWatch alarms created by the pattern.
dynamoTable dynamodb.Table Returns an instance of the DynamoDB table created by the pattern.
elasticsearchDomain elasticsearch.CfnDomain Returns an instance of the Elasticsearch domain created by the pattern.
identityPool cognito.CfnIdentityPool Returns an instance of the Cognito identity pool created by the pattern.
lambdaFunction lambda.Function Returns an instance of the Lambda function created by the pattern.
userPool cognito.UserPool Returns an instance of the Cognito user pool created by the pattern.
userPoolClient cognito.UserPoolClient Returns an instance of the Cognito user pool client created by the pattern.

Lambda function

This pattern requires a Lambda function that can post data into the Elasticsearch service from the DynamoDB stream. A sample function is provided here.

Default settings

Out-of-the-box implementation of this pattern without any overrides will set the following defaults:

Amazon DynamoDB Table

  • Set the billing mode for DynamoDB Table to On-Demand (Pay per request)

  • Enable server-side encryption for DynamoDB Table using AWS managed KMS Key

  • Creates a partition key called 'id' for DynamoDB Table

  • Retain the Table when deleting the CloudFormation stack

  • Enable continuous backups and point-in-time recovery

AWS Lambda Function

  • Configure limited privilege access IAM role for Lambda function

  • Enable reusing connections with Keep-Alive for NodeJs Lambda function

  • Enable X-Ray tracing

  • Enable Failure-Handling features: enable bisect on function Error; set default Maximum Record Age (24 hours); set default Maximum Retry Attempts (500); and deploy SQS dead-letter queue as destination on failure

  • Set environment variables:

    • AWS_NODEJS_CONNECTION_REUSE_ENABLED (for Node 10.x and higher functions)

Amazon Cognito

  • Set password policy for User Pools

  • Enforce the advanced security mode for User Pools

Amazon Elasticsearch Service

  • Deploy best practices CloudWatch Alarms for the Elasticsearch Domain

  • Secure the Kibana dashboard access with Cognito User Pools

  • Enable server-side encryption for Elasticsearch Domain using AWS managed KMS Key

  • Enable node-to-node encryption for Elasticsearch Domain

  • Configure the cluster for the Amazon ES domain

Architecture

GitHub

To view the code for this pattern, create/view issues and pull requests, and more:
@aws-solutions-constructs/aws-dynamodb-stream-lambda-elasticsearch-kibana