S3Table L2 construct
A Glue table that targets a S3 dataset.
Example
declare const myDatabase: glue.Database;
new glue.S3Table(this, 'MyTable', {
database: myDatabase,
columns: [{
name: 'data',
type: glue.Schema.STRING,
}],
partitionKeys: [{
name: 'date',
type: glue.Schema.STRING,
}],
dataFormat: glue.DataFormat.JSON,
partitionProjection: {
date: glue.PartitionProjectionConfiguration.date({
min: 'NOW-3YEARS',
max: 'NOW',
format: 'yyyy-MM-dd',
}),
},
});
# my_database: glue.Database
glue.S3Table(self, "MyTable",
database=my_database,
columns=[glue.Column(
name="data",
type=glue.Schema.STRING
)],
partition_keys=[glue.Column(
name="date",
type=glue.Schema.STRING
)],
data_format=glue.DataFormat.JSON,
partition_projection={
"date": glue.PartitionProjectionConfiguration.date(
min="NOW-3YEARS",
max="NOW",
format="yyyy-MM-dd"
)
}
)
Database myDatabase;
S3Table.Builder.create(this, "MyTable")
.database(myDatabase)
.columns(List.of(Column.builder()
.name("data")
.type(Schema.STRING)
.build()))
.partitionKeys(List.of(Column.builder()
.name("date")
.type(Schema.STRING)
.build()))
.dataFormat(DataFormat.JSON)
.partitionProjection(Map.of(
"date", PartitionProjectionConfiguration.date(DatePartitionProjectionConfigurationProps.builder()
.min("NOW-3YEARS")
.max("NOW")
.format("yyyy-MM-dd")
.build())))
.build();
// Install: dotnet add package Amazon.CDK.Lib
Database myDatabase;
new S3Table(this, "MyTable", new S3TableProps {
Database = myDatabase,
Columns = new [] { new Column {
Name = "data",
Type = Schema.STRING
} },
PartitionKeys = new [] { new Column {
Name = "date",
Type = Schema.STRING
} },
DataFormat = DataFormat.JSON,
PartitionProjection = new Dictionary<string, PartitionProjectionConfiguration> {
{ "date", PartitionProjectionConfiguration.Date(new DatePartitionProjectionConfigurationProps {
Min = "NOW-3YEARS",
Max = "NOW",
Format = "yyyy-MM-dd"
}) }
}
});
// Install: go get github.com/aws/aws-cdk-go/awscdk/v2
var myDatabase Database
glue.NewS3Table(this, jsii.String("MyTable"), &S3TableProps{
Database: myDatabase,
Columns: []Column{
&Column{
Name: jsii.String("data"),
Type: glue.Schema_STRING(),
},
},
PartitionKeys: []Column{
&Column{
Name: jsii.String("date"),
Type: glue.Schema_STRING(),
},
},
DataFormat: glue.DataFormat_JSON(),
PartitionProjection: map[string]PartitionProjectionConfiguration{
"date": glue.PartitionProjectionConfiguration_date(&DatePartitionProjectionConfigurationProps{
"min": jsii.String("NOW-3YEARS"),
"max": jsii.String("NOW"),
"format": jsii.String("yyyy-MM-dd"),
}),
},
})
Initializer
new S3Table(scope: Construct, id: string, props: S3TableProps)
S3Table(scope: Construct, id: str, props: S3TableProps)
new S3Table(Construct scope, String id, S3TableProps props)
new S3Table(Construct scope, string id, S3TableProps props)
NewS3Table(Scope Construct, Id *string, Props S3TableProps)
Construct Props
clientSideEncryptionTableClientSideEncryptionclient_side_encryptionTableClientSideEncryptionclientSideEncryptionTableClientSideEncryptionClientSideEncryptionTableClientSideEncryptionClientSideEncryptionTableClientSideEncryption= no client-side encryption#Independent of the bucket's server-side encryption, and valid whether the bucket is managed or provided.
columnsColumn[]columnstyping.List[Column]columnsList<Column>ColumnsColumn[]Columns*[]Columnrequired#dataFormatDataFormatdata_formatDataFormatdataFormatDataFormatDataFormatDataFormatDataFormatDataFormatrequired#enablePartitionFilteringbooleanenable_partition_filteringboolenablePartitionFilteringBooleanEnablePartitionFilteringboolEnablePartitionFiltering*bool= The parameter is not defined#hasEncryptedDatabooleanhas_encrypted_databoolhasEncryptedDataBooleanHasEncryptedDataboolHasEncryptedData*bool= true#notes
This sets the has_encrypted_data table parameter. Athena reads it when querying client-side (CSE-KMS) encrypted datasets; for server-side encrypted (SSE-S3 / SSE-KMS) or unencrypted data it has no effect, since Amazon S3 decrypts server-side encrypted objects transparently.
Do not also set has_encrypted_data through parameters - use this property instead. A conflicting value in parameters is rejected.
parameters{ [key: string]: string }parameterstyping.Mapping[str, str]parametersMap<String, String>ParametersIDictionary<string, string>Parameters*map[string]*string= The parameter is not defined#The key/value pairs that are allowed to be submitted are not limited, however their functionality is not guaranteed.
partitionIndexesPartitionIndex[]partition_indexestyping.List[PartitionIndex]partitionIndexesList<PartitionIndex>PartitionIndexesPartitionIndex[]PartitionIndexes*[]PartitionIndex= table has no partition indexes#A maximum of 3 indexes are allowed on a table. Keys in the index must be part of the table's partition keys.
partitionKeysColumn[]partition_keystyping.List[Column]partitionKeysList<Column>PartitionKeysColumn[]PartitionKeys*[]Column= table is not partitioned#partitionProjection{ [key: string]: PartitionProjectionConfiguration }partition_projectiontyping.Mapping[str, PartitionProjectionConfiguration]partitionProjectionMap<String, PartitionProjectionConfiguration>PartitionProjectionIDictionary<string, PartitionProjectionConfiguration>PartitionProjection*map[string]PartitionProjectionConfiguration= No partition projection#Partition projection allows Athena to automatically add new partitions without requiring ALTER TABLE ADD PARTITION statements.
s3Prefixstrings3_prefixstrs3PrefixStringS3PrefixstringS3Prefix*string= No prefix. The data will be stor…#When the table shares a bucket with other tables or consumers, set this so that the grant* methods scope S3 access to this table's data. Without a prefix, those grants cover the entire bucket.
default
Default: No prefix. The data will be stored under the root of the bucket.
storageS3TableStoragestorageS3TableStoragestorageS3TableStorageStorageS3TableStorageStorageS3TableStorage= a managed bucket with S3-managed…#default
Default: a managed bucket with S3-managed (SSE-S3) encryption
storageParametersStorageParameter[]storage_parameterstyping.List[StorageParameter]storageParametersList<StorageParameter>StorageParametersStorageParameter[]StorageParameters*[]StorageParameter= The parameter is not defined#notes · example
These properties help describe the format of the data that is stored within the crawled data sources.
The key/value pairs that are allowed to be submitted are not limited, however their functionality is not guaranteed.
Some keys will be auto-populated by glue crawlers, however, you can override them by specifying the key and value in this property.
declare const glueDatabase: glue.IDatabase;
const table = new glue.S3Table(this, 'Table', {
storageParameters: [
glue.StorageParameter.skipHeaderLineCount(1),
glue.StorageParameter.compressionType(glue.CompressionType.GZIP),
glue.StorageParameter.custom('foo', 'bar'), // Will have no effect
glue.StorageParameter.custom('separatorChar', ','), // Will describe the separator char used in the data
glue.StorageParameter.custom(glue.StorageParameters.WRITE_PARALLEL, 'off'),
],
// ...
database: glueDatabase,
columns: [{
name: 'col1',
type: glue.Schema.STRING,
}],
dataFormat: glue.DataFormat.CSV,
});
storedAsSubDirectoriesbooleanstored_as_sub_directoriesboolstoredAsSubDirectoriesBooleanStoredAsSubDirectoriesboolStoredAsSubDirectories*bool= false#Properties
clientSideEncryptionKeyIKeyRefclient_side_encryption_keyIKeyRefclientSideEncryptionKeyIKeyRefClientSideEncryptionKeyIKeyRefClientSideEncryptionKeyIKeyRef#For server-side (bucket) encryption, read bucket.encryptionKey instead.
columnsColumn[]columnstyping.List[Column]columnsList<Column>ColumnsColumn[]Columns*[]Columnrequired#dataFormatDataFormatdata_formatDataFormatdataFormatDataFormatDataFormatDataFormatDataFormatDataFormatrequired#envResourceEnvironmentenvResourceEnvironmentenvResourceEnvironmentEnvResourceEnvironmentEnvResourceEnvironmentrequired#notes
For resources that are created and managed in a Stack (those created by creating new class instances like new Role(), new Bucket(), etc.), this is always the same as the environment of the stack they belong to.
For referenced resources (those obtained from referencing methods like Role.fromRoleArn(), Bucket.fromBucketName(), etc.), they might be different than the stack they were imported into.
partitionIndexesPartitionIndex[]partition_indexestyping.List[PartitionIndex]partitionIndexesList<PartitionIndex>PartitionIndexesPartitionIndex[]PartitionIndexes*[]PartitionIndex#partitionKeysColumn[]partition_keystyping.List[Column]partitionKeysList<Column>PartitionKeysColumn[]PartitionKeys*[]Column#partitionProjection{ [key: string]: PartitionProjectionConfiguration }partition_projectiontyping.Mapping[str, PartitionProjectionConfiguration]partitionProjectionMap<String, PartitionProjectionConfiguration>PartitionProjectionIDictionary<string, PartitionProjectionConfiguration>PartitionProjection*map[string]PartitionProjectionConfiguration#PROPERTY_INJECTION_IDstringproperty_injection_idstrPROPERTY_INJECTION_IDStringPROPERTY_INJECTION_IDstringPROPERTY_INJECTION_ID*stringrequiredstatic#storageParametersStorageParameter[]storage_parameterstyping.List[StorageParameter]storageParametersList<StorageParameter>StorageParametersStorageParameter[]StorageParameters*[]StorageParameter#Methods
addPartitionIndex(index: PartitionIndex): voiddef add_partition_index(self, index: PartitionIndex) -> Nonepublic void addPartitionIndex(PartitionIndex index)public void AddPartitionIndex(PartitionIndex index)func AddPartitionIndex(Index PartitionIndex)#You can have a maximum of 3 partition indexes to a table. Partition index keys must be a subset of the table's partition keys.
applyCrossStackReferenceStrength(strength: ReferenceStrength): voiddef apply_cross_stack_reference_strength(self, strength: ReferenceStrength) -> Nonepublic void applyCrossStackReferenceStrength(ReferenceStrength strength)public void ApplyCrossStackReferenceStrength(ReferenceStrength strength)func ApplyCrossStackReferenceStrength(Strength ReferenceStrength)#notes · 1 parameter
When set, any cross-stack reference to this resource will use the specified mechanism instead of the global default determined by the @aws-cdk/core:defaultCrossStackReferences context key. This is useful for selectively weakening specific references to avoid the "deadly embrace" problem without changing the app-wide default.
Parameters
strengthstrengthstrengthstrengthStrength— - The reference strength to use for this resource.
applyRemovalPolicy(policy: RemovalPolicy): voiddef apply_removal_policy(self, policy: RemovalPolicy) -> Nonepublic void applyRemovalPolicy(RemovalPolicy policy)public void ApplyRemovalPolicy(RemovalPolicy policy)func ApplyRemovalPolicy(Policy RemovalPolicy)#notes
The Removal Policy controls what happens to this resource when it stops being managed by CloudFormation, either because you've removed it from the CDK application or because you've made a change that requires the resource to be replaced.
The resource can be deleted (RemovalPolicy.DESTROY), or left in your AWS account for data recovery and cleanup later (RemovalPolicy.RETAIN).
fromTableArn(scope: Construct, id: string, tableArn: string): ITable@staticmethod def from_table_arn(scope: Construct, id: str, table_arn: str) -> ITablepublic static ITable fromTableArn(Construct scope, String id, String tableArn)public static ITable FromTableArn(Construct scope, string id, string tableArn)func FromTableArn(Scope Construct, Id *string, TableArn *string) ITablestatic#fromTableAttributes(scope: Construct, id: string, attrs: TableAttributes): ITable@staticmethod def from_table_attributes(scope: Construct, id: str, attrs: TableAttributes) -> ITablepublic static ITable fromTableAttributes(Construct scope, String id, TableAttributes attrs)public static ITable FromTableAttributes(Construct scope, string id, TableAttributes attrs)func FromTableAttributes(Scope Construct, Id *string, Attrs TableAttributes) ITablestatic#3 parameters
Parameters
scopescopescopescopeScope— The scope creating construct (usuallythis).ididididId— The construct's id.attrsattrsattrsattrsAttrs— Import attributes.
grant(grantee: IGrantable, actions: string[]): Grantdef grant(self, grantee: IGrantable, actions: typing.List[str]) -> Grantpublic Grant grant(IGrantable grantee, List<String> actions)public Grant Grant(IGrantable grantee, string[] actions)func Grant(Grantee IGrantable, Actions *[]*string) Grant#notes · 2 parameters
This is a low-level escape hatch: the actions are applied verbatim, scoped to this table's ARN. Prefer the intent-based grantRead / grantWrite / grantReadWrite methods, which grant a curated set of actions and also cover the underlying S3 data. Only pass the specific actions the grantee needs - avoid service wildcards such as glue:*. [disable-awslint:no-grants]
Parameters
granteegranteegranteegranteeGrantee— the principal.actionsactionsactionsactionsActions— the set of Glue actions to allow (for exampleglue:GetTable).
grantRead(grantee: IGrantable): Grantdef grant_read(self, grantee: IGrantable) -> Grantpublic Grant grantRead(IGrantable grantee)public Grant GrantRead(IGrantable grantee)func GrantRead(Grantee IGrantable) Grant#[disable-awslint:no-grants]
1 parameter
Parameters
granteegranteegranteegranteeGrantee— the principal.
grantReadWrite(grantee: IGrantable): Grantdef grant_read_write(self, grantee: IGrantable) -> Grantpublic Grant grantReadWrite(IGrantable grantee)public Grant GrantReadWrite(IGrantable grantee)func GrantReadWrite(Grantee IGrantable) Grant#[disable-awslint:no-grants]
1 parameter
Parameters
granteegranteegranteegranteeGrantee— the principal.
grantToUnderlyingResources(grantee: IGrantable, actions: string[]): Grantdef grant_to_underlying_resources(self, grantee: IGrantable, actions: typing.List[str]) -> Grantpublic Grant grantToUnderlyingResources(IGrantable grantee, List<String> actions)public Grant GrantToUnderlyingResources(IGrantable grantee, string[] actions)func GrantToUnderlyingResources(Grantee IGrantable, Actions *[]*string) Grant#notes · 2 parameters
This is a low-level escape hatch for actions (such as certain Lake Formation or crawler operations) that must be authorized against the catalog and database in addition to the table. The actions are applied verbatim to all three ARNs (table, catalog, database), so scope them tightly: pass only the specific actions the grantee needs and avoid service wildcards such as glue:*, which would grant broad access across every resource in the catalog and database. [disable-awslint:no-grants]
Parameters
granteegranteegranteegranteeGrantee— the principal.actionsactionsactionsactionsActions— the set of Glue actions to allow (for exampleglue:GetTable).
grantWrite(grantee: IGrantable): Grantdef grant_write(self, grantee: IGrantable) -> Grantpublic Grant grantWrite(IGrantable grantee)public Grant GrantWrite(IGrantable grantee)func GrantWrite(Grantee IGrantable) Grant#[disable-awslint:no-grants]
1 parameter
Parameters
granteegranteegranteegranteeGrantee— the principal.
isConstruct(x: any): boolean@staticmethod def is_construct(x: Any) -> boolpublic static Boolean isConstruct(Object x)public static bool IsConstruct(object x)func IsConstruct(X interface{}) *boolstatic#notes · 1 parameter · returns
Use this method instead of instanceof to properly detect Construct instances, even when the construct library is symlinked.
Explanation: in JavaScript, multiple copies of the constructs library on disk are seen as independent, completely different libraries. As a consequence, the class Construct in each copy of the constructs library is seen as a different class, and an instance of one class will not test as instanceof the other class. npm install will not create installations like this, but users may manually symlink construct libraries together or use a monorepo tool: in those cases, multiple copies of the constructs library can be accidentally installed, and instanceof will behave unpredictably. It is safest to avoid using instanceof, and using this type-testing method instead.
Parameters
xxxxX— Any object.
Returns: true if x is an object created from a class which extends Construct.
isOwnedResource(construct: IConstruct): boolean@staticmethod def is_owned_resource(construct: IConstruct) -> boolpublic static Boolean isOwnedResource(IConstruct construct)public static bool IsOwnedResource(IConstruct construct)func IsOwnedResource(Construct IConstruct) *boolstatic#isResource(construct: IConstruct): boolean@staticmethod def is_resource(construct: IConstruct) -> boolpublic static Boolean isResource(IConstruct construct)public static bool IsResource(IConstruct construct)func IsResource(Construct IConstruct) *boolstatic#toString(): stringdef to_string(self, ) -> strpublic String toString()public string ToString()func ToString() *string#with(...mixins: IMixin[]): IConstructdef with(self, mixins: IMixin) -> IConstructpublic IConstruct with(IMixin mixins)public IConstruct With(IMixin mixins)func With(Mixins IMixin) IConstruct#Mixins are applied in order. The list of constructs is captured at the start of the call, so constructs added by a mixin will not be visited. Use multiple with() calls if subsequent mixins should apply to added constructs.