Namespace Amazon.CDK.AWS.Redshift.Alpha

Amazon Redshift Construct Library

--- cdk-constructs: Experimental 
The APIs of higher level constructs in this module are experimental and under active development.
They are subject to non-backward compatible changes or removal in any future version. These are
not subject to the <a href="https://semver.org/">Semantic Versioning</a> model and breaking changes will be
announced in the release notes. This means that while you may use them, you may need to update
your source code when upgrading to a newer version of this package.

Starting a Redshift Cluster Database

To set up a Redshift cluster, define a Cluster. It will be launched in a VPC. You can specify a VPC, otherwise one will be created. The nodes are always launched in private subnets and are encrypted by default.

using Amazon.CDK.AWS.EC2;


var vpc = new Vpc(this, "Vpc");
var cluster = new Cluster(this, "Redshift", new ClusterProps {
    MasterUser = new Login {
        MasterUsername = "admin"
    },
    Vpc = vpc
});

By default, the master password will be generated and stored in AWS Secrets Manager. You can specify characters to not include in generated passwords by setting excludeCharacters property.

using Amazon.CDK.AWS.EC2;


var vpc = new Vpc(this, "Vpc");
var cluster = new Cluster(this, "Redshift", new ClusterProps {
    MasterUser = new Login {
        MasterUsername = "admin",
        ExcludeCharacters = "\"@/\\ '`"
    },
    Vpc = vpc
});

A default database named default_db will be created in the cluster. To change the name of this database set the defaultDatabaseName attribute in the constructor properties.

By default, the cluster will not be publicly accessible. Depending on your use case, you can make the cluster publicly accessible with the publiclyAccessible property.

Adding a logging bucket for database audit logging to S3

Amazon Redshift logs information about connections and user activities in your database. These logs help you to monitor the database for security and troubleshooting purposes, a process called database auditing. To send these logs to an S3 bucket, specify the loggingProperties when creating a new cluster.

using Amazon.CDK.AWS.EC2;
using Amazon.CDK.AWS.S3;


var vpc = new Vpc(this, "Vpc");
var bucket = Bucket.FromBucketName(this, "bucket", "logging-bucket");

var cluster = new Cluster(this, "Redshift", new ClusterProps {
    MasterUser = new Login {
        MasterUsername = "admin"
    },
    Vpc = vpc,
    LoggingProperties = new LoggingProperties {
        LoggingBucket = bucket,
        LoggingKeyPrefix = "prefix"
    }
});

Connecting

To control who can access the cluster, use the .connections attribute. Redshift Clusters have a default port, so you don't need to specify the port:

cluster.Connections.AllowDefaultPortFromAnyIpv4("Open to the world");

The endpoint to access your database cluster will be available as the .clusterEndpoint attribute:

cluster.ClusterEndpoint.SocketAddress;

Database Resources

This module allows for the creation of non-CloudFormation database resources such as users and tables. This allows you to manage identities, permissions, and stateful resources within your Redshift cluster from your CDK application.

Because these resources are not available in CloudFormation, this library leverages custom resources to manage them. In addition to the IAM permissions required to make Redshift service calls, the execution role for the custom resource handler requires database credentials to create resources within the cluster.

These database credentials can be supplied explicitly through the adminUser properties of the various database resource constructs. Alternatively, the credentials can be automatically pulled from the Redshift cluster's default administrator credentials. However, this option is only available if the password for the credentials was generated by the CDK application (ie., no value vas provided for the masterPassword property of Cluster.masterUser).

Creating Users

Create a user within a Redshift cluster database by instantiating a User construct. This will generate a username and password, store the credentials in a AWS Secrets Manager Secret, and make a query to the Redshift cluster to create a new database user with the credentials.

new User(this, "User", new UserProps {
    Cluster = cluster,
    DatabaseName = "databaseName"
});

By default, the user credentials are encrypted with your AWS account's default Secrets Manager encryption key. You can specify the encryption key used for this purpose by supplying a key in the encryptionKey property.

using Amazon.CDK.AWS.KMS;


var encryptionKey = new Key(this, "Key");
new User(this, "User", new UserProps {
    EncryptionKey = encryptionKey,
    Cluster = cluster,
    DatabaseName = "databaseName"
});

By default, a username is automatically generated from the user construct ID and its path in the construct tree. You can specify a particular username by providing a value for the username property. Usernames must be valid identifiers; see: Names and identifiers in the Amazon Redshift Database Developer Guide.

new User(this, "User", new UserProps {
    Username = "myuser",
    Cluster = cluster,
    DatabaseName = "databaseName"
});

The user password is generated by AWS Secrets Manager using the default configuration found in secretsmanager.SecretStringGenerator, except with password length 30 and some SQL-incompliant characters excluded. The plaintext for the password will never be present in the CDK application; instead, a CloudFormation Dynamic Reference will be used wherever the password value is required.

You can specify characters to not include in generated passwords by setting excludeCharacters property.

new User(this, "User", new UserProps {
    Cluster = cluster,
    DatabaseName = "databaseName",
    ExcludeCharacters = "\"@/\\ '`"
});

Creating Tables

Create a table within a Redshift cluster database by instantiating a Table construct. This will make a query to the Redshift cluster to create a new database table with the supplied schema.

new Table(this, "Table", new TableProps {
    TableColumns = new [] { new Column { Name = "col1", DataType = "varchar(4)" }, new Column { Name = "col2", DataType = "float" } },
    Cluster = cluster,
    DatabaseName = "databaseName"
});

Tables greater than v2.114.1 can have their table name changed, for versions <= v2.114.1, this would not be possible. Therefore, changing of table names for <= v2.114.1 have been disabled.

// Example automatically generated from non-compiling source. May contain errors.
new Table(this, "Table", new TableProps {
    TableName = "oldTableName",  // This value can be change for versions greater than v2.114.1
    TableColumns = new [] { new Column { Name = "col1", DataType = "varchar(4)" }, new Column { Name = "col2", DataType = "float" } },
    Cluster = cluster,
    DatabaseName = "databaseName"
});

The table can be configured to have distStyle attribute and a distKey column:

new Table(this, "Table", new TableProps {
    TableColumns = new [] { new Column { Name = "col1", DataType = "varchar(4)", DistKey = true }, new Column { Name = "col2", DataType = "float" } },
    Cluster = cluster,
    DatabaseName = "databaseName",
    DistStyle = TableDistStyle.KEY
});

The table can also be configured to have sortStyle attribute and sortKey columns:

new Table(this, "Table", new TableProps {
    TableColumns = new [] { new Column { Name = "col1", DataType = "varchar(4)", SortKey = true }, new Column { Name = "col2", DataType = "float", SortKey = true } },
    Cluster = cluster,
    DatabaseName = "databaseName",
    SortStyle = TableSortStyle.COMPOUND
});

Tables and their respective columns can be configured to contain comments:

new Table(this, "Table", new TableProps {
    TableColumns = new [] { new Column { Name = "col1", DataType = "varchar(4)", Comment = "This is a column comment" }, new Column { Name = "col2", DataType = "float", Comment = "This is a another column comment" } },
    Cluster = cluster,
    DatabaseName = "databaseName",
    TableComment = "This is a table comment"
});

Table columns can be configured to use a specific compression encoding:

using Amazon.CDK.AWS.Redshift.Alpha;


new Table(this, "Table", new TableProps {
    TableColumns = new [] { new Column { Name = "col1", DataType = "varchar(4)", Encoding = ColumnEncoding.TEXT32K }, new Column { Name = "col2", DataType = "float", Encoding = ColumnEncoding.DELTA32K } },
    Cluster = cluster,
    DatabaseName = "databaseName"
});

Table columns can also contain an id attribute, which can allow table columns to be renamed.

NOTE To use the id attribute, you must also enable the @aws-cdk/aws-redshift:columnId feature flag.

new Table(this, "Table", new TableProps {
    TableColumns = new [] { new Column { Id = "col1", Name = "col1", DataType = "varchar(4)" }, new Column { Id = "col2", Name = "col2", DataType = "float" } },
    Cluster = cluster,
    DatabaseName = "databaseName"
});

Granting Privileges

You can give a user privileges to perform certain actions on a table by using the Table.grant() method.

var user = new User(this, "User", new UserProps {
    Cluster = cluster,
    DatabaseName = "databaseName"
});
var table = new Table(this, "Table", new TableProps {
    TableColumns = new [] { new Column { Name = "col1", DataType = "varchar(4)" }, new Column { Name = "col2", DataType = "float" } },
    Cluster = cluster,
    DatabaseName = "databaseName"
});

table.Grant(user, TableAction.DROP, TableAction.SELECT);

Take care when managing privileges via the CDK, as attempting to manage a user's privileges on the same table in multiple CDK applications could lead to accidentally overriding these permissions. Consider the following two CDK applications which both refer to the same user and table. In application 1, the resources are created and the user is given INSERT permissions on the table:

var databaseName = "databaseName";
var username = "myuser";
var tableName = "mytable";

var user = new User(this, "User", new UserProps {
    Username = username,
    Cluster = cluster,
    DatabaseName = databaseName
});
var table = new Table(this, "Table", new TableProps {
    TableColumns = new [] { new Column { Name = "col1", DataType = "varchar(4)" }, new Column { Name = "col2", DataType = "float" } },
    Cluster = cluster,
    DatabaseName = databaseName
});
table.Grant(user, TableAction.INSERT);

In application 2, the resources are imported and the user is given INSERT permissions on the table:

var databaseName = "databaseName";
var username = "myuser";
var tableName = "mytable";

var user = User.FromUserAttributes(this, "User", new UserAttributes {
    Username = username,
    Password = SecretValue.UnsafePlainText("NOT_FOR_PRODUCTION"),
    Cluster = cluster,
    DatabaseName = databaseName
});
var table = Table.FromTableAttributes(this, "Table", new TableAttributes {
    TableName = tableName,
    TableColumns = new [] { new Column { Name = "col1", DataType = "varchar(4)" }, new Column { Name = "col2", DataType = "float" } },
    Cluster = cluster,
    DatabaseName = "databaseName"
});
table.Grant(user, TableAction.INSERT);

Both applications attempt to grant the user the appropriate privilege on the table by submitting a GRANT USER SQL query to the Redshift cluster. Note that the latter of these two calls will have no effect since the user has already been granted the privilege.

Now, if application 1 were to remove the call to grant, a REVOKE USER SQL query is submitted to the Redshift cluster. In general, application 1 does not know that application 2 has also granted this permission and thus cannot decide not to issue the revocation. This leads to the undesirable state where application 2 still contains the call to grant but the user does not have the specified permission.

Note that this does not occur when duplicate privileges are granted within the same application, as such privileges are de-duplicated before any SQL query is submitted.

Rotating credentials

When the master password is generated and stored in AWS Secrets Manager, it can be rotated automatically:

cluster.AddRotationSingleUser();

The multi user rotation scheme is also available:

var user = new User(this, "User", new UserProps {
    Cluster = cluster,
    DatabaseName = "databaseName"
});
cluster.AddRotationMultiUser("MultiUserRotation", new RotationMultiUserOptions {
    Secret = user.Secret
});

Adding Parameters

You can add a parameter to a parameter group withClusterParameterGroup.addParameter().

using Amazon.CDK.AWS.Redshift.Alpha;


var params = new ClusterParameterGroup(this, "Params", new ClusterParameterGroupProps {
    Description = "desc",
    Parameters = new Dictionary<string, string> {
        { "require_ssl", "true" }
    }
});

params.AddParameter("enable_user_activity_logging", "true");

Additionally, you can add a parameter to the cluster's associated parameter group with Cluster.addToParameterGroup(). If the cluster does not have an associated parameter group, a new parameter group is created.

using Amazon.CDK.AWS.EC2;
using Amazon.CDK;
Vpc vpc;

var cluster = new Cluster(this, "Cluster", new ClusterProps {
    MasterUser = new Login {
        MasterUsername = "admin",
        MasterPassword = SecretValue.UnsafePlainText("tooshort")
    },
    Vpc = vpc
});

cluster.AddToParameterGroup("enable_user_activity_logging", "true");

Rebooting for Parameter Updates

In most cases, existing clusters must be manually rebooted to apply parameter changes. You can automate parameter related reboots by setting the cluster's rebootForParameterChanges property to true , or by using Cluster.enableRebootForParameterChanges().

using Amazon.CDK.AWS.EC2;
using Amazon.CDK;
Vpc vpc;

var cluster = new Cluster(this, "Cluster", new ClusterProps {
    MasterUser = new Login {
        MasterUsername = "admin",
        MasterPassword = SecretValue.UnsafePlainText("tooshort")
    },
    Vpc = vpc
});

cluster.AddToParameterGroup("enable_user_activity_logging", "true");
cluster.EnableRebootForParameterChanges();

Elastic IP

If you configure your cluster to be publicly accessible, you can optionally select an elastic IP address to use for the external IP address. An elastic IP address is a static IP address that is associated with your AWS account. You can use an elastic IP address to connect to your cluster from outside the VPC. An elastic IP address gives you the ability to change your underlying configuration without affecting the IP address that clients use to connect to your cluster. This approach can be helpful for situations such as recovery after a failure.

using Amazon.CDK.AWS.EC2;
using Amazon.CDK;
Vpc vpc;

new Cluster(this, "Redshift", new ClusterProps {
    MasterUser = new Login {
        MasterUsername = "admin",
        MasterPassword = SecretValue.UnsafePlainText("tooshort")
    },
    Vpc = vpc,
    PubliclyAccessible = true,
    ElasticIp = "10.123.123.255"
});

If the Cluster is in a VPC and you want to connect to it using the private IP address from within the cluster, it is important to enable DNS resolution and DNS hostnames in the VPC config. If these parameters would not be set, connections from within the VPC would connect to the elastic IP address and not the private IP address.

using Amazon.CDK.AWS.EC2;

var vpc = new Vpc(this, "VPC", new VpcProps {
    EnableDnsSupport = true,
    EnableDnsHostnames = true
});

Note that if there is already an existing, public accessible Cluster, which VPC configuration is changed to use DNS hostnames and DNS resolution, connections still use the elastic IP address until the cluster is resized.

Elastic IP vs. Cluster node public IP

The elastic IP address is an external IP address for accessing the cluster outside of a VPC. It's not related to the cluster node public IP addresses and private IP addresses that are accessible via the clusterEndpoint property. The public and private cluster node IP addresses appear regardless of whether the cluster is publicly accessible or not. They are used only in certain circumstances to configure ingress rules on the remote host. These circumstances occur when you load data from an Amazon EC2 instance or other remote host using a Secure Shell (SSH) connection.

Attach Elastic IP after Cluster creation

In some cases, you might want to associate the cluster with an elastic IP address or change an elastic IP address that is associated with the cluster. To attach an elastic IP address after the cluster is created, first update the cluster so that it is not publicly accessible, then make it both publicly accessible and add an Elastic IP address in the same operation.

Enhanced VPC Routing

When you use Amazon Redshift enhanced VPC routing, Amazon Redshift forces all COPY and UNLOAD traffic between your cluster and your data repositories through your virtual private cloud (VPC) based on the Amazon VPC service. By using enhanced VPC routing, you can use standard VPC features, such as VPC security groups, network access control lists (ACLs), VPC endpoints, VPC endpoint policies, internet gateways, and Domain Name System (DNS) servers, as described in the Amazon VPC User Guide. You use these features to tightly manage the flow of data between your Amazon Redshift cluster and other resources. When you use enhanced VPC routing to route traffic through your VPC, you can also use VPC flow logs to monitor COPY and UNLOAD traffic.

using Amazon.CDK.AWS.EC2;
using Amazon.CDK;
Vpc vpc;

new Cluster(this, "Redshift", new ClusterProps {
    MasterUser = new Login {
        MasterUsername = "admin",
        MasterPassword = SecretValue.UnsafePlainText("tooshort")
    },
    Vpc = vpc,
    EnhancedVpcRouting = true
});

If enhanced VPC routing is not enabled, Amazon Redshift routes traffic through the internet, including traffic to other services within the AWS network.

Default IAM role

Some Amazon Redshift features require Amazon Redshift to access other AWS services on your behalf. For your Amazon Redshift clusters to act on your behalf, you supply security credentials to your clusters. The preferred method to supply security credentials is to specify an AWS Identity and Access Management (IAM) role.

When you create an IAM role and set it as the default for the cluster using console, you don't have to provide the IAM role's Amazon Resource Name (ARN) to perform authentication and authorization.

using Amazon.CDK.AWS.EC2;
using Amazon.CDK.AWS.IAM;
Vpc vpc;


var defaultRole = new Role(this, "DefaultRole", new RoleProps {
    AssumedBy = new ServicePrincipal("redshift.amazonaws.com")
});

new Cluster(this, "Redshift", new ClusterProps {
    MasterUser = new Login {
        MasterUsername = "admin"
    },
    Vpc = vpc,
    Roles = new [] { defaultRole },
    DefaultRole = defaultRole
});

A default role can also be added to a cluster using the addDefaultIamRole method.

using Amazon.CDK.AWS.EC2;
using Amazon.CDK.AWS.IAM;
Vpc vpc;


var defaultRole = new Role(this, "DefaultRole", new RoleProps {
    AssumedBy = new ServicePrincipal("redshift.amazonaws.com")
});

var redshiftCluster = new Cluster(this, "Redshift", new ClusterProps {
    MasterUser = new Login {
        MasterUsername = "admin"
    },
    Vpc = vpc,
    Roles = new [] { defaultRole }
});

redshiftCluster.AddDefaultIamRole(defaultRole);

IAM roles

Attaching IAM roles to a Redshift Cluster grants permissions to the Redshift service to perform actions on your behalf.

using Amazon.CDK.AWS.EC2;
using Amazon.CDK.AWS.IAM;
Vpc vpc;


var role = new Role(this, "Role", new RoleProps {
    AssumedBy = new ServicePrincipal("redshift.amazonaws.com")
});
var cluster = new Cluster(this, "Redshift", new ClusterProps {
    MasterUser = new Login {
        MasterUsername = "admin"
    },
    Vpc = vpc,
    Roles = new [] { role }
});

Additional IAM roles can be attached to a cluster using the addIamRole method.

using Amazon.CDK.AWS.EC2;
using Amazon.CDK.AWS.IAM;
Vpc vpc;


var role = new Role(this, "Role", new RoleProps {
    AssumedBy = new ServicePrincipal("redshift.amazonaws.com")
});
var cluster = new Cluster(this, "Redshift", new ClusterProps {
    MasterUser = new Login {
        MasterUsername = "admin"
    },
    Vpc = vpc
});
cluster.AddIamRole(role);

Multi-AZ

Amazon Redshift supports multiple Availability Zones (Multi-AZ) deployments for provisioned RA3 clusters. By using Multi-AZ deployments, your Amazon Redshift data warehouse can continue operating in failure scenarios when an unexpected event happens in an Availability Zone.

To create a Multi-AZ cluster, set the multiAz property to true when creating the cluster.

// Example automatically generated from non-compiling source. May contain errors.
ec2.IVpc vpc;


new redshift.Cluster(stack, "Cluster", new Struct {
    MasterUser = new Struct {
        MasterUsername = "admin"
    },
    Vpc = vpc,  // 3 AZs are required for Multi-AZ
    NodeType = redshift.NodeType.RA3_XLPLUS,  // must be RA3 node type
    ClusterType = redshift.ClusterType.MULTI_NODE,  // must be MULTI_NODE
    NumberOfNodes = 2,  // must be 2 or more
    MultiAz = true
});

Resizing

As your data warehousing needs change, it's possible to resize your Redshift cluster. If the cluster was deployed via CDK, it's important to resize it via CDK so the change is registered in the AWS CloudFormation template. There are two types of resize operations:

    There are other constraints to be aware of, for example, elastic resizing does not support single-node clusters and there are limits on the number of nodes you can add to a cluster. See the AWS Redshift Documentation and AWS API Documentation for more details.

    Classes

    Cluster

    (experimental) Create a Redshift cluster a given number of nodes.
    ClusterAttributes

    (experimental) Properties that describe an existing cluster instance.
    ClusterParameterGroup

    (experimental) A cluster parameter group.
    ClusterParameterGroupProps

    (experimental) Properties for a parameter group.
    ClusterProps

    (experimental) Properties for a new database cluster.
    ClusterSubnetGroup

    (experimental) Class for creating a Redshift cluster subnet group.
    ClusterSubnetGroupProps

    (experimental) Properties for creating a ClusterSubnetGroup.
    ClusterType

    (experimental) What cluster type to use.
    Column

    (experimental) A column in a Redshift table.
    ColumnEncoding

    (experimental) The compression encoding of a column.
    DatabaseOptions

    (experimental) Properties for accessing a Redshift database.
    DatabaseSecret

    (experimental) A database secret.
    DatabaseSecretProps

    (experimental) Construction properties for a DatabaseSecret.
    Endpoint

    (experimental) Connection endpoint of a redshift cluster.
    LoggingProperties

    (experimental) Logging bucket and S3 prefix combination.
    Login

    (experimental) Username and password combination.
    NodeType

    (experimental) Possible Node Types to use in the cluster used for defining ClusterProps.nodeType.
    RotationMultiUserOptions

    (experimental) Options to add the multi user rotation.
    Table

    (experimental) A table in a Redshift cluster.
    TableAction

    (experimental) An action that a Redshift user can be granted privilege to perform on a table.
    TableAttributes

    (experimental) A full specification of a Redshift table that can be used to import it fluently into the CDK application.
    TableDistStyle

    (experimental) The data distribution style of a table.
    TableProps

    (experimental) Properties for configuring a Redshift table.
    TableSortStyle

    (experimental) The sort style of a table.
    User

    (experimental) A user in a Redshift cluster.
    UserAttributes

    (experimental) A full specification of a Redshift user that can be used to import it fluently into the CDK application.
    UserProps

    (experimental) Properties for configuring a Redshift user.

    Interfaces

    ICluster

    (experimental) Create a Redshift Cluster with a given number of nodes.
    IClusterAttributes

    (experimental) Properties that describe an existing cluster instance.
    IClusterParameterGroup

    (experimental) A parameter group.
    IClusterParameterGroupProps

    (experimental) Properties for a parameter group.
    IClusterProps

    (experimental) Properties for a new database cluster.
    IClusterSubnetGroup

    (experimental) Interface for a cluster subnet group.
    IClusterSubnetGroupProps

    (experimental) Properties for creating a ClusterSubnetGroup.
    IColumn

    (experimental) A column in a Redshift table.
    IDatabaseOptions

    (experimental) Properties for accessing a Redshift database.
    IDatabaseSecretProps

    (experimental) Construction properties for a DatabaseSecret.
    ILoggingProperties

    (experimental) Logging bucket and S3 prefix combination.
    ILogin

    (experimental) Username and password combination.
    IRotationMultiUserOptions

    (experimental) Options to add the multi user rotation.
    ITable

    (experimental) Represents a table in a Redshift database.
    ITableAttributes

    (experimental) A full specification of a Redshift table that can be used to import it fluently into the CDK application.
    ITableProps

    (experimental) Properties for configuring a Redshift table.
    IUser

    (experimental) Represents a user in a Redshift database.
    IUserAttributes

    (experimental) A full specification of a Redshift user that can be used to import it fluently into the CDK application.
    IUserProps

    (experimental) Properties for configuring a Redshift user.