Namespace Amazon.CDK.AWS.Glue
AWS Glue Construct Library
---AWS CDK v1 has reached End-of-Support on 2023-06-01.
This package is no longer being updated, and users should migrate to AWS CDK v2.
For more information on how to migrate, see the Migrating to AWS CDK v2 guide.
This module is part of the AWS Cloud Development Kit project.
Job
A Job
encapsulates a script that connects to data sources, processes them, and then writes output to a data target.
There are 3 types of jobs supported by AWS Glue: Spark ETL, Spark Streaming, and Python Shell jobs.
The glue.JobExecutable
allows you to specify the type of job, the language to use and the code assets required by the job.
glue.Code
allows you to refer to the different code assets required by the job, either from an existing S3 location or from a local file path.
Spark Jobs
These jobs run in an Apache Spark environment managed by AWS Glue.
ETL Jobs
An ETL job processes data in batches using Apache Spark.
Bucket bucket;
new Job(this, "ScalaSparkEtlJob", new JobProps {
Executable = JobExecutable.ScalaEtl(new ScalaJobExecutableProps {
GlueVersion = GlueVersion.V2_0,
Script = Code.FromBucket(bucket, "src/com/example/HelloWorld.scala"),
ClassName = "com.example.HelloWorld",
ExtraJars = new [] { Code.FromBucket(bucket, "jars/HelloWorld.jar") }
}),
Description = "an example Scala ETL job"
});
Streaming Jobs
A Streaming job is similar to an ETL job, except that it performs ETL on data streams. It uses the Apache Spark Structured Streaming framework. Some Spark job features are not available to streaming ETL jobs.
new Job(this, "PythonSparkStreamingJob", new JobProps {
Executable = JobExecutable.PythonStreaming(new PythonSparkJobExecutableProps {
GlueVersion = GlueVersion.V2_0,
PythonVersion = PythonVersion.THREE,
Script = Code.FromAsset(Join(__dirname, "job-script/hello_world.py"))
}),
Description = "an example Python Streaming job"
});
Python Shell Jobs
A Python shell job runs Python scripts as a shell and supports a Python version that depends on the AWS Glue version you are using. This can be used to schedule and run tasks that don't require an Apache Spark environment.
Bucket bucket;
new Job(this, "PythonShellJob", new JobProps {
Executable = JobExecutable.PythonShell(new PythonShellExecutableProps {
GlueVersion = GlueVersion.V1_0,
PythonVersion = PythonVersion.THREE,
Script = Code.FromBucket(bucket, "script.py")
}),
Description = "an example Python Shell job"
});
See documentation for more information on adding jobs in Glue.
Connection
A Connection
allows Glue jobs, crawlers and development endpoints to access certain types of data stores. For example, to create a network connection to connect to a data source within a VPC:
SecurityGroup securityGroup;
Subnet subnet;
new Connection(this, "MyConnection", new ConnectionProps {
Type = ConnectionType.NETWORK,
// The security groups granting AWS Glue inbound access to the data source within the VPC
SecurityGroups = new [] { securityGroup },
// The VPC subnet which contains the data source
Subnet = subnet
});
If you need to use a connection type that doesn't exist as a static member on ConnectionType
, you can instantiate a ConnectionType
object, e.g: new glue.ConnectionType('NEW_TYPE')
.
See Adding a Connection to Your Data Store and Connection Structure documentation for more information on the supported data stores and their configurations.
SecurityConfiguration
A SecurityConfiguration
is a set of security properties that can be used by AWS Glue to encrypt data at rest.
new SecurityConfiguration(this, "MySecurityConfiguration", new SecurityConfigurationProps {
SecurityConfigurationName = "name",
CloudWatchEncryption = new CloudWatchEncryption {
Mode = CloudWatchEncryptionMode.KMS
},
JobBookmarksEncryption = new JobBookmarksEncryption {
Mode = JobBookmarksEncryptionMode.CLIENT_SIDE_KMS
},
S3Encryption = new S3Encryption {
Mode = S3EncryptionMode.KMS
}
});
By default, a shared KMS key is created for use with the encryption configurations that require one. You can also supply your own key for each encryption config, for example, for CloudWatch encryption:
Key key;
new SecurityConfiguration(this, "MySecurityConfiguration", new SecurityConfigurationProps {
SecurityConfigurationName = "name",
CloudWatchEncryption = new CloudWatchEncryption {
Mode = CloudWatchEncryptionMode.KMS,
KmsKey = key
}
});
See documentation for more info for Glue encrypting data written by Crawlers, Jobs, and Development Endpoints.
Database
A Database
is a logical grouping of Tables
in the Glue Catalog.
new Database(this, "MyDatabase", new DatabaseProps {
DatabaseName = "my_database"
});
Table
A Glue table describes a table of data in S3: its structure (column names and types), location of data (S3 objects with a common prefix in a S3 bucket), and format for the files (Json, Avro, Parquet, etc.):
Database myDatabase;
new Table(this, "MyTable", new TableProps {
Database = myDatabase,
TableName = "my_table",
Columns = new [] { new Column {
Name = "col1",
Type = Schema.STRING
}, new Column {
Name = "col2",
Type = Schema.Array(Schema.STRING),
Comment = "col2 is an array of strings"
} },
DataFormat = DataFormat.JSON
});
By default, a S3 bucket will be created to store the table's data but you can manually pass the bucket
and s3Prefix
:
Bucket myBucket;
Database myDatabase;
new Table(this, "MyTable", new TableProps {
Bucket = myBucket,
S3Prefix = "my-table/",
// ...
Database = myDatabase,
TableName = "my_table",
Columns = new [] { new Column {
Name = "col1",
Type = Schema.STRING
} },
DataFormat = DataFormat.JSON
});
By default, an S3 bucket will be created to store the table's data and stored in the bucket root. You can also manually pass the bucket
and s3Prefix
:
Partition Keys
To improve query performance, a table can specify partitionKeys
on which data is stored and queried separately. For example, you might partition a table by year
and month
to optimize queries based on a time window:
Database myDatabase;
new Table(this, "MyTable", new TableProps {
Database = myDatabase,
TableName = "my_table",
Columns = new [] { new Column {
Name = "col1",
Type = Schema.STRING
} },
PartitionKeys = new [] { new Column {
Name = "year",
Type = Schema.SMALL_INT
}, new Column {
Name = "month",
Type = Schema.SMALL_INT
} },
DataFormat = DataFormat.JSON
});
Partition Indexes
Another way to improve query performance is to specify partition indexes. If no partition indexes are present on the table, AWS Glue loads all partitions of the table and filters the loaded partitions using the query expression. The query takes more time to run as the number of partitions increase. With an index, the query will try to fetch a subset of the partitions instead of loading all partitions of the table.
The keys of a partition index must be a subset of the partition keys of the table. You can have a
maximum of 3 partition indexes per table. To specify a partition index, you can use the partitionIndexes
property:
Database myDatabase;
new Table(this, "MyTable", new TableProps {
Database = myDatabase,
TableName = "my_table",
Columns = new [] { new Column {
Name = "col1",
Type = Schema.STRING
} },
PartitionKeys = new [] { new Column {
Name = "year",
Type = Schema.SMALL_INT
}, new Column {
Name = "month",
Type = Schema.SMALL_INT
} },
PartitionIndexes = new [] { new PartitionIndex {
IndexName = "my-index", // optional
KeyNames = new [] { "year" }
} }, // supply up to 3 indexes
DataFormat = DataFormat.JSON
});
Alternatively, you can call the addPartitionIndex()
function on a table:
Table myTable;
myTable.AddPartitionIndex(new PartitionIndex {
IndexName = "my-index",
KeyNames = new [] { "year" }
});
Encryption
You can enable encryption on a Table's data:
Database myDatabase;
new Table(this, "MyTable", new TableProps {
Encryption = TableEncryption.S3_MANAGED,
// ...
Database = myDatabase,
TableName = "my_table",
Columns = new [] { new Column {
Name = "col1",
Type = Schema.STRING
} },
DataFormat = DataFormat.JSON
});
Database myDatabase;
// KMS key is created automatically
// KMS key is created automatically
new Table(this, "MyTable", new TableProps {
Encryption = TableEncryption.KMS,
// ...
Database = myDatabase,
TableName = "my_table",
Columns = new [] { new Column {
Name = "col1",
Type = Schema.STRING
} },
DataFormat = DataFormat.JSON
});
// with an explicit KMS key
// with an explicit KMS key
new Table(this, "MyTable", new TableProps {
Encryption = TableEncryption.KMS,
EncryptionKey = new Key(this, "MyKey"),
// ...
Database = myDatabase,
TableName = "my_table",
Columns = new [] { new Column {
Name = "col1",
Type = Schema.STRING
} },
DataFormat = DataFormat.JSON
});
Database myDatabase;
new Table(this, "MyTable", new TableProps {
Encryption = TableEncryption.KMS_MANAGED,
// ...
Database = myDatabase,
TableName = "my_table",
Columns = new [] { new Column {
Name = "col1",
Type = Schema.STRING
} },
DataFormat = DataFormat.JSON
});
Database myDatabase;
// KMS key is created automatically
// KMS key is created automatically
new Table(this, "MyTable", new TableProps {
Encryption = TableEncryption.CLIENT_SIDE_KMS,
// ...
Database = myDatabase,
TableName = "my_table",
Columns = new [] { new Column {
Name = "col1",
Type = Schema.STRING
} },
DataFormat = DataFormat.JSON
});
// with an explicit KMS key
// with an explicit KMS key
new Table(this, "MyTable", new TableProps {
Encryption = TableEncryption.CLIENT_SIDE_KMS,
EncryptionKey = new Key(this, "MyKey"),
// ...
Database = myDatabase,
TableName = "my_table",
Columns = new [] { new Column {
Name = "col1",
Type = Schema.STRING
} },
DataFormat = DataFormat.JSON
});
Note: you cannot provide a Bucket
when creating the Table
if you wish to use server-side encryption (KMS
, KMS_MANAGED
or S3_MANAGED
).
Types
A table's schema is a collection of columns, each of which have a name
and a type
. Types are recursive structures, consisting of primitive and complex types:
Database myDatabase;
new Table(this, "MyTable", new TableProps {
Columns = new [] { new Column {
Name = "primitive_column",
Type = Schema.STRING
}, new Column {
Name = "array_column",
Type = Schema.Array(Schema.INTEGER),
Comment = "array<integer>"
}, new Column {
Name = "map_column",
Type = Schema.Map(Schema.STRING, Schema.TIMESTAMP),
Comment = "map<string,string>"
}, new Column {
Name = "struct_column",
Type = Schema.Struct(new [] { new Column {
Name = "nested_column",
Type = Schema.DATE,
Comment = "nested comment"
} }),
Comment = "struct<nested_column:date COMMENT 'nested comment'>"
} },
// ...
Database = myDatabase,
TableName = "my_table",
DataFormat = DataFormat.JSON
});
Primitives
Numeric
Name | Type | Comments |
---|---|---|
FLOAT | Constant | A 32-bit single-precision floating point number |
INTEGER | Constant | A 32-bit signed value in two's complement format, with a minimum value of -2^31 and a maximum value of 2^31-1 |
DOUBLE | Constant | A 64-bit double-precision floating point number |
BIG_INT | Constant | A 64-bit signed INTEGER in two’s complement format, with a minimum value of -2^63 and a maximum value of 2^63 -1 |
SMALL_INT | Constant | A 16-bit signed INTEGER in two’s complement format, with a minimum value of -2^15 and a maximum value of 2^15-1 |
TINY_INT | Constant | A 8-bit signed INTEGER in two’s complement format, with a minimum value of -2^7 and a maximum value of 2^7-1 |
Date and time
Name | Type | Comments |
---|---|---|
DATE | Constant | A date in UNIX format, such as YYYY-MM-DD. |
TIMESTAMP | Constant | Date and time instant in the UNiX format, such as yyyy-mm-dd hh:mm:ss[.f...]. For example, TIMESTAMP '2008-09-15 03:04:05.324'. This format uses the session time zone. |
String
Name | Type | Comments |
---|---|---|
STRING | Constant | A string literal enclosed in single or double quotes |
decimal(precision: number, scale?: number) | Function | precision is the total number of digits. scale (optional) is the number of digits in fractional part with a default of 0. For example, use these type definitions: decimal(11,5), decimal(15) |
char(length: number) | Function | Fixed length character data, with a specified length between 1 and 255, such as char(10) |
varchar(length: number) | Function | Variable length character data, with a specified length between 1 and 65535, such as varchar(10) |
Miscellaneous
Name | Type | Comments |
---|---|---|
BOOLEAN | Constant | Values are true and false |
BINARY | Constant | Value is in binary |
Complex
Name | Type | Comments |
---|---|---|
array(itemType: Type) | Function | An array of some other type |
map(keyType: Type, valueType: Type) | Function | A map of some primitive key type to any value type |
struct(collumns: Column[]) | Function | Nested structure containing individually named and typed collumns |
Classes
AssetCode | (experimental) Job Code from a local file. |
CfnClassifier | A CloudFormation |
CfnClassifier.CsvClassifierProperty | A classifier for custom |
CfnClassifier.GrokClassifierProperty | A classifier that uses |
CfnClassifier.JsonClassifierProperty | A classifier for |
CfnClassifier.XMLClassifierProperty | A classifier for |
CfnClassifierProps | Properties for defining a |
CfnConnection | A CloudFormation |
CfnConnection.ConnectionInputProperty | A structure that is used to specify a connection to create or update. |
CfnConnection.PhysicalConnectionRequirementsProperty | Specifies the physical requirements for a connection. |
CfnConnectionProps | Properties for defining a |
CfnCrawler | A CloudFormation |
CfnCrawler.CatalogTargetProperty | Specifies an AWS Glue Data Catalog target. |
CfnCrawler.DeltaTargetProperty | |
CfnCrawler.DynamoDBTargetProperty | Specifies an Amazon DynamoDB table to crawl. |
CfnCrawler.JdbcTargetProperty | Specifies a JDBC data store to crawl. |
CfnCrawler.MongoDBTargetProperty | Specifies an Amazon DocumentDB or MongoDB data store to crawl. |
CfnCrawler.RecrawlPolicyProperty | When crawling an Amazon S3 data source after the first crawl is complete, specifies whether to crawl the entire dataset again or to crawl only folders that were added since the last crawler run. |
CfnCrawler.S3TargetProperty | Specifies a data store in Amazon Simple Storage Service (Amazon S3). |
CfnCrawler.ScheduleProperty | A scheduling object using a |
CfnCrawler.SchemaChangePolicyProperty | The policy that specifies update and delete behaviors for the crawler. |
CfnCrawler.TargetsProperty | Specifies data stores to crawl. |
CfnCrawlerProps | Properties for defining a |
CfnDatabase | A CloudFormation |
CfnDatabase.DatabaseIdentifierProperty | A structure that describes a target database for resource linking. |
CfnDatabase.DatabaseInputProperty | The structure used to create or update a database. |
CfnDatabase.DataLakePrincipalProperty | The AWS Lake Formation principal. |
CfnDatabase.FederatedDatabaseProperty | |
CfnDatabase.PrincipalPrivilegesProperty | the permissions granted to a principal. |
CfnDatabaseProps | Properties for defining a |
CfnDataCatalogEncryptionSettings | A CloudFormation |
CfnDataCatalogEncryptionSettings.ConnectionPasswordEncryptionProperty | The data structure used by the Data Catalog to encrypt the password as part of |
CfnDataCatalogEncryptionSettings.DataCatalogEncryptionSettingsProperty | Contains configuration information for maintaining Data Catalog security. |
CfnDataCatalogEncryptionSettings.EncryptionAtRestProperty | Specifies the encryption-at-rest configuration for the Data Catalog. |
CfnDataCatalogEncryptionSettingsProps | Properties for defining a |
CfnDevEndpoint | A CloudFormation |
CfnDevEndpointProps | Properties for defining a |
CfnJob | A CloudFormation |
CfnJob.ConnectionsListProperty | Specifies the connections used by a job. |
CfnJob.ExecutionPropertyProperty | An execution property of a job. |
CfnJob.JobCommandProperty | Specifies code executed when a job is run. |
CfnJob.NotificationPropertyProperty | Specifies configuration properties of a notification. |
CfnJobProps | Properties for defining a |
CfnMLTransform | A CloudFormation |
CfnMLTransform.FindMatchesParametersProperty | The parameters to configure the find matches transform. |
CfnMLTransform.GlueTablesProperty | The database and table in the AWS Glue Data Catalog that is used for input or output data. |
CfnMLTransform.InputRecordTablesProperty | A list of AWS Glue table definitions used by the transform. |
CfnMLTransform.MLUserDataEncryptionProperty | The encryption-at-rest settings of the transform that apply to accessing user data. |
CfnMLTransform.TransformEncryptionProperty | The encryption-at-rest settings of the transform that apply to accessing user data. |
CfnMLTransform.TransformParametersProperty | The algorithm-specific parameters that are associated with the machine learning transform. |
CfnMLTransformProps | Properties for defining a |
CfnPartition | A CloudFormation |
CfnPartition.ColumnProperty | A column in a |
CfnPartition.OrderProperty | Specifies the sort order of a sorted column. |
CfnPartition.PartitionInputProperty | The structure used to create and update a partition. |
CfnPartition.SchemaIdProperty | A structure that contains schema identity fields. |
CfnPartition.SchemaReferenceProperty | An object that references a schema stored in the AWS Glue Schema Registry. |
CfnPartition.SerdeInfoProperty | Information about a serialization/deserialization program (SerDe) that serves as an extractor and loader. |
CfnPartition.SkewedInfoProperty | Specifies skewed values in a table. |
CfnPartition.StorageDescriptorProperty | Describes the physical storage of table data. |
CfnPartitionProps | Properties for defining a |
CfnRegistry | A CloudFormation |
CfnRegistryProps | Properties for defining a |
CfnSchema | A CloudFormation |
CfnSchema.RegistryProperty | Specifies a registry in the AWS Glue Schema Registry. |
CfnSchema.SchemaVersionProperty | Specifies the version of a schema. |
CfnSchemaProps | Properties for defining a |
CfnSchemaVersion | A CloudFormation |
CfnSchemaVersion.SchemaProperty | A wrapper structure to contain schema identity fields. |
CfnSchemaVersionMetadata | A CloudFormation |
CfnSchemaVersionMetadataProps | Properties for defining a |
CfnSchemaVersionProps | Properties for defining a |
CfnSecurityConfiguration | A CloudFormation |
CfnSecurityConfiguration.CloudWatchEncryptionProperty | Specifies how Amazon CloudWatch data should be encrypted. |
CfnSecurityConfiguration.EncryptionConfigurationProperty | Specifies an encryption configuration. |
CfnSecurityConfiguration.JobBookmarksEncryptionProperty | Specifies how job bookmark data should be encrypted. |
CfnSecurityConfiguration.S3EncryptionProperty | Specifies how Amazon Simple Storage Service (Amazon S3) data should be encrypted. |
CfnSecurityConfigurationProps | Properties for defining a |
CfnTable | A CloudFormation |
CfnTable.ColumnProperty | A column in a |
CfnTable.OrderProperty | Specifies the sort order of a sorted column. |
CfnTable.SchemaIdProperty | A structure that contains schema identity fields. |
CfnTable.SchemaReferenceProperty | An object that references a schema stored in the AWS Glue Schema Registry. |
CfnTable.SerdeInfoProperty | Information about a serialization/deserialization program (SerDe) that serves as an extractor and loader. |
CfnTable.SkewedInfoProperty | Specifies skewed values in a table. |
CfnTable.StorageDescriptorProperty | Describes the physical storage of table data. |
CfnTable.TableIdentifierProperty | A structure that describes a target table for resource linking. |
CfnTable.TableInputProperty | A structure used to define a table. |
CfnTableProps | Properties for defining a |
CfnTrigger | A CloudFormation |
CfnTrigger.ActionProperty | Defines an action to be initiated by a trigger. |
CfnTrigger.ConditionProperty | Defines a condition under which a trigger fires. |
CfnTrigger.EventBatchingConditionProperty | Batch condition that must be met (specified number of events received or batch time window expired) before EventBridge event trigger fires. |
CfnTrigger.NotificationPropertyProperty | Specifies configuration properties of a job run notification. |
CfnTrigger.PredicateProperty | Defines the predicate of the trigger, which determines when it fires. |
CfnTriggerProps | Properties for defining a |
CfnWorkflow | A CloudFormation |
CfnWorkflowProps | Properties for defining a |
ClassificationString | (experimental) Classification string given to tables with this data format. |
CloudWatchEncryption | (experimental) CloudWatch Logs encryption configuration. |
CloudWatchEncryptionMode | (experimental) Encryption mode for CloudWatch Logs. |
Code | (experimental) Represents a Glue Job's Code assets (an asset can be a scripts, a jar, a python file or any other file). |
CodeConfig | (experimental) Result of binding |
Column | (experimental) A column of a table. |
Connection | (experimental) An AWS Glue connection to a data source. |
ConnectionOptions | (experimental) Base Connection Options. |
ConnectionProps | (experimental) Construction properties for {@link Connection}. |
ConnectionType | (experimental) The type of the glue connection. |
ContinuousLoggingProps | (experimental) Properties for enabling Continuous Logging for Glue Jobs. |
Database | (experimental) A Glue database. |
DatabaseProps | |
DataFormat | (experimental) Defines the input/output formats and ser/de for a single DataFormat. |
DataFormatProps | (experimental) Properties of a DataFormat instance. |
GlueVersion | (experimental) AWS Glue version determines the versions of Apache Spark and Python that are available to the job. |
InputFormat | (experimental) Absolute class name of the Hadoop |
Job | (experimental) A Glue Job. |
JobAttributes | (experimental) Attributes for importing {@link Job}. |
JobBookmarksEncryption | (experimental) Job bookmarks encryption configuration. |
JobBookmarksEncryptionMode | (experimental) Encryption mode for Job Bookmarks. |
JobExecutable | (experimental) The executable properties related to the Glue job's GlueVersion, JobType and code. |
JobExecutableConfig | (experimental) Result of binding a |
JobLanguage | (experimental) Runtime language of the Glue job. |
JobProps | (experimental) Construction properties for {@link Job}. |
JobState | (experimental) Job states emitted by Glue to CloudWatch Events. |
JobType | (experimental) The job type. |
MetricType | (experimental) The Glue CloudWatch metric type. |
OutputFormat | (experimental) Absolute class name of the Hadoop |
PartitionIndex | (experimental) Properties of a Partition Index. |
PythonShellExecutableProps | (experimental) Props for creating a Python shell job executable. |
PythonSparkJobExecutableProps | (experimental) Props for creating a Python Spark (ETL or Streaming) job executable. |
PythonVersion | (experimental) Python version. |
S3Code | (experimental) Glue job Code from an S3 bucket. |
S3Encryption | (experimental) S3 encryption configuration. |
S3EncryptionMode | (experimental) Encryption mode for S3. |
ScalaJobExecutableProps | (experimental) Props for creating a Scala Spark (ETL or Streaming) job executable. |
Schema | |
SecurityConfiguration | (experimental) A security configuration is a set of security properties that can be used by AWS Glue to encrypt data at rest. |
SecurityConfigurationProps | (experimental) Constructions properties of {@link SecurityConfiguration}. |
SerializationLibrary | (experimental) Serialization library to use when serializing/deserializing (SerDe) table records. |
SparkUILoggingLocation | (experimental) The Spark UI logging location. |
SparkUIProps | (experimental) Properties for enabling Spark UI monitoring feature for Spark-based Glue jobs. |
Table | (experimental) A Glue table. |
TableAttributes | |
TableEncryption | (experimental) Encryption options for a Table. |
TableProps | |
Type | (experimental) Represents a type of a column in a table schema. |
WorkerType | (experimental) The type of predefined worker that is allocated when a job runs. |
Interfaces
CfnClassifier.ICsvClassifierProperty | A classifier for custom |
CfnClassifier.IGrokClassifierProperty | A classifier that uses |
CfnClassifier.IJsonClassifierProperty | A classifier for |
CfnClassifier.IXMLClassifierProperty | A classifier for |
CfnConnection.IConnectionInputProperty | A structure that is used to specify a connection to create or update. |
CfnConnection.IPhysicalConnectionRequirementsProperty | Specifies the physical requirements for a connection. |
CfnCrawler.ICatalogTargetProperty | Specifies an AWS Glue Data Catalog target. |
CfnCrawler.IDeltaTargetProperty | |
CfnCrawler.IDynamoDBTargetProperty | Specifies an Amazon DynamoDB table to crawl. |
CfnCrawler.IJdbcTargetProperty | Specifies a JDBC data store to crawl. |
CfnCrawler.IMongoDBTargetProperty | Specifies an Amazon DocumentDB or MongoDB data store to crawl. |
CfnCrawler.IRecrawlPolicyProperty | When crawling an Amazon S3 data source after the first crawl is complete, specifies whether to crawl the entire dataset again or to crawl only folders that were added since the last crawler run. |
CfnCrawler.IS3TargetProperty | Specifies a data store in Amazon Simple Storage Service (Amazon S3). |
CfnCrawler.IScheduleProperty | A scheduling object using a |
CfnCrawler.ISchemaChangePolicyProperty | The policy that specifies update and delete behaviors for the crawler. |
CfnCrawler.ITargetsProperty | Specifies data stores to crawl. |
CfnDatabase.IDatabaseIdentifierProperty | A structure that describes a target database for resource linking. |
CfnDatabase.IDatabaseInputProperty | The structure used to create or update a database. |
CfnDatabase.IDataLakePrincipalProperty | The AWS Lake Formation principal. |
CfnDatabase.IFederatedDatabaseProperty | |
CfnDatabase.IPrincipalPrivilegesProperty | the permissions granted to a principal. |
CfnDataCatalogEncryptionSettings.IConnectionPasswordEncryptionProperty | The data structure used by the Data Catalog to encrypt the password as part of |
CfnDataCatalogEncryptionSettings.IDataCatalogEncryptionSettingsProperty | Contains configuration information for maintaining Data Catalog security. |
CfnDataCatalogEncryptionSettings.IEncryptionAtRestProperty | Specifies the encryption-at-rest configuration for the Data Catalog. |
CfnJob.IConnectionsListProperty | Specifies the connections used by a job. |
CfnJob.IExecutionPropertyProperty | An execution property of a job. |
CfnJob.IJobCommandProperty | Specifies code executed when a job is run. |
CfnJob.INotificationPropertyProperty | Specifies configuration properties of a notification. |
CfnMLTransform.IFindMatchesParametersProperty | The parameters to configure the find matches transform. |
CfnMLTransform.IGlueTablesProperty | The database and table in the AWS Glue Data Catalog that is used for input or output data. |
CfnMLTransform.IInputRecordTablesProperty | A list of AWS Glue table definitions used by the transform. |
CfnMLTransform.IMLUserDataEncryptionProperty | The encryption-at-rest settings of the transform that apply to accessing user data. |
CfnMLTransform.ITransformEncryptionProperty | The encryption-at-rest settings of the transform that apply to accessing user data. |
CfnMLTransform.ITransformParametersProperty | The algorithm-specific parameters that are associated with the machine learning transform. |
CfnPartition.IColumnProperty | A column in a |
CfnPartition.IOrderProperty | Specifies the sort order of a sorted column. |
CfnPartition.IPartitionInputProperty | The structure used to create and update a partition. |
CfnPartition.ISchemaIdProperty | A structure that contains schema identity fields. |
CfnPartition.ISchemaReferenceProperty | An object that references a schema stored in the AWS Glue Schema Registry. |
CfnPartition.ISerdeInfoProperty | Information about a serialization/deserialization program (SerDe) that serves as an extractor and loader. |
CfnPartition.ISkewedInfoProperty | Specifies skewed values in a table. |
CfnPartition.IStorageDescriptorProperty | Describes the physical storage of table data. |
CfnSchema.IRegistryProperty | Specifies a registry in the AWS Glue Schema Registry. |
CfnSchema.ISchemaVersionProperty | Specifies the version of a schema. |
CfnSchemaVersion.ISchemaProperty | A wrapper structure to contain schema identity fields. |
CfnSecurityConfiguration.ICloudWatchEncryptionProperty | Specifies how Amazon CloudWatch data should be encrypted. |
CfnSecurityConfiguration.IEncryptionConfigurationProperty | Specifies an encryption configuration. |
CfnSecurityConfiguration.IJobBookmarksEncryptionProperty | Specifies how job bookmark data should be encrypted. |
CfnSecurityConfiguration.IS3EncryptionProperty | Specifies how Amazon Simple Storage Service (Amazon S3) data should be encrypted. |
CfnTable.IColumnProperty | A column in a |
CfnTable.IOrderProperty | Specifies the sort order of a sorted column. |
CfnTable.ISchemaIdProperty | A structure that contains schema identity fields. |
CfnTable.ISchemaReferenceProperty | An object that references a schema stored in the AWS Glue Schema Registry. |
CfnTable.ISerdeInfoProperty | Information about a serialization/deserialization program (SerDe) that serves as an extractor and loader. |
CfnTable.ISkewedInfoProperty | Specifies skewed values in a table. |
CfnTable.IStorageDescriptorProperty | Describes the physical storage of table data. |
CfnTable.ITableIdentifierProperty | A structure that describes a target table for resource linking. |
CfnTable.ITableInputProperty | A structure used to define a table. |
CfnTrigger.IActionProperty | Defines an action to be initiated by a trigger. |
CfnTrigger.IConditionProperty | Defines a condition under which a trigger fires. |
CfnTrigger.IEventBatchingConditionProperty | Batch condition that must be met (specified number of events received or batch time window expired) before EventBridge event trigger fires. |
CfnTrigger.INotificationPropertyProperty | Specifies configuration properties of a job run notification. |
CfnTrigger.IPredicateProperty | Defines the predicate of the trigger, which determines when it fires. |
ICfnClassifierProps | Properties for defining a |
ICfnConnectionProps | Properties for defining a |
ICfnCrawlerProps | Properties for defining a |
ICfnDatabaseProps | Properties for defining a |
ICfnDataCatalogEncryptionSettingsProps | Properties for defining a |
ICfnDevEndpointProps | Properties for defining a |
ICfnJobProps | Properties for defining a |
ICfnMLTransformProps | Properties for defining a |
ICfnPartitionProps | Properties for defining a |
ICfnRegistryProps | Properties for defining a |
ICfnSchemaProps | Properties for defining a |
ICfnSchemaVersionMetadataProps | Properties for defining a |
ICfnSchemaVersionProps | Properties for defining a |
ICfnSecurityConfigurationProps | Properties for defining a |
ICfnTableProps | Properties for defining a |
ICfnTriggerProps | Properties for defining a |
ICfnWorkflowProps | Properties for defining a |
ICloudWatchEncryption | (experimental) CloudWatch Logs encryption configuration. |
ICodeConfig | (experimental) Result of binding |
IColumn | (experimental) A column of a table. |
IConnection | (experimental) Interface representing a created or an imported {@link Connection}. |
IConnectionOptions | (experimental) Base Connection Options. |
IConnectionProps | (experimental) Construction properties for {@link Connection}. |
IContinuousLoggingProps | (experimental) Properties for enabling Continuous Logging for Glue Jobs. |
IDatabase | |
IDatabaseProps | |
IDataFormatProps | (experimental) Properties of a DataFormat instance. |
IJob | (experimental) Interface representing a created or an imported {@link Job}. |
IJobAttributes | (experimental) Attributes for importing {@link Job}. |
IJobBookmarksEncryption | (experimental) Job bookmarks encryption configuration. |
IJobExecutableConfig | (experimental) Result of binding a |
IJobProps | (experimental) Construction properties for {@link Job}. |
IPartitionIndex | (experimental) Properties of a Partition Index. |
IPythonShellExecutableProps | (experimental) Props for creating a Python shell job executable. |
IPythonSparkJobExecutableProps | (experimental) Props for creating a Python Spark (ETL or Streaming) job executable. |
IS3Encryption | (experimental) S3 encryption configuration. |
IScalaJobExecutableProps | (experimental) Props for creating a Scala Spark (ETL or Streaming) job executable. |
ISecurityConfiguration | (experimental) Interface representing a created or an imported {@link SecurityConfiguration}. |
ISecurityConfigurationProps | (experimental) Constructions properties of {@link SecurityConfiguration}. |
ISparkUILoggingLocation | (experimental) The Spark UI logging location. |
ISparkUIProps | (experimental) Properties for enabling Spark UI monitoring feature for Spark-based Glue jobs. |
ITable | |
ITableAttributes | |
ITableProps | |
IType | (experimental) Represents a type of a column in a table schema. |