AWS Glue DataBrew 2017-07-25
- Client: Aws\GlueDataBrew\GlueDataBrewClient
- Service ID: databrew
- Version: 2017-07-25
This page describes the parameters and results for the operations of the AWS Glue DataBrew (2017-07-25), and shows how to use the Aws\GlueDataBrew\GlueDataBrewClient object to call the described operations. This documentation is specific to the 2017-07-25 API version of the service.
Operation Summary
Each of the following operations can be created from a client using
$client->getCommand('CommandName'), where "CommandName" is the
name of one of the following operations. Note: a command is a value that
encapsulates an operation and the parameters used to create an HTTP request.
You can also create and send a command immediately using the magic methods
available on a client object: $client->commandName(/* parameters */).
You can send the command asynchronously (returning a promise) by appending the
word "Async" to the operation name: $client->commandNameAsync(/* parameters */).
- BatchDeleteRecipeVersion ( array $params = [] )
Deletes one or more versions of a recipe at a time.
- CreateDataset ( array $params = [] )
Creates a new DataBrew dataset.
- CreateProfileJob ( array $params = [] )
Creates a new job to analyze a dataset and create its data profile.
- CreateProject ( array $params = [] )
Creates a new DataBrew project.
- CreateRecipe ( array $params = [] )
Creates a new DataBrew recipe.
- CreateRecipeJob ( array $params = [] )
Creates a new job to transform input data, using steps defined in an existing AWS Glue DataBrew recipe
- CreateSchedule ( array $params = [] )
Creates a new schedule for one or more DataBrew jobs.
- DeleteDataset ( array $params = [] )
Deletes a dataset from DataBrew.
- DeleteJob ( array $params = [] )
Deletes the specified DataBrew job.
- DeleteProject ( array $params = [] )
Deletes an existing DataBrew project.
- DeleteRecipeVersion ( array $params = [] )
Deletes a single version of a DataBrew recipe.
- DeleteSchedule ( array $params = [] )
Deletes the specified DataBrew schedule.
- DescribeDataset ( array $params = [] )
Returns the definition of a specific DataBrew dataset.
- DescribeJob ( array $params = [] )
Returns the definition of a specific DataBrew job.
- DescribeJobRun ( array $params = [] )
Represents one run of a DataBrew job.
- DescribeProject ( array $params = [] )
Returns the definition of a specific DataBrew project.
- DescribeRecipe ( array $params = [] )
Returns the definition of a specific DataBrew recipe corresponding to a particular version.
- DescribeSchedule ( array $params = [] )
Returns the definition of a specific DataBrew schedule.
- ListDatasets ( array $params = [] )
Lists all of the DataBrew datasets.
- ListJobRuns ( array $params = [] )
Lists all of the previous runs of a particular DataBrew job.
- ListJobs ( array $params = [] )
Lists all of the DataBrew jobs that are defined.
- ListProjects ( array $params = [] )
Lists all of the DataBrew projects that are defined.
- ListRecipeVersions ( array $params = [] )
Lists the versions of a particular DataBrew recipe, except for LATEST_WORKING.
- ListRecipes ( array $params = [] )
Lists all of the DataBrew recipes that are defined.
- ListSchedules ( array $params = [] )
Lists the DataBrew schedules that are defined.
- ListTagsForResource ( array $params = [] )
Lists all the tags for a DataBrew resource.
- PublishRecipe ( array $params = [] )
Publishes a new version of a DataBrew recipe.
- SendProjectSessionAction ( array $params = [] )
Performs a recipe step within an interactive DataBrew session that's currently open.
- StartJobRun ( array $params = [] )
Runs a DataBrew job.
- StartProjectSession ( array $params = [] )
Creates an interactive session, enabling you to manipulate data in a DataBrew project.
- StopJobRun ( array $params = [] )
Stops a particular run of a job.
- TagResource ( array $params = [] )
Adds metadata tags to a DataBrew resource, such as a dataset, project, recipe, job, or schedule.
- UntagResource ( array $params = [] )
Removes metadata tags from a DataBrew resource.
- UpdateDataset ( array $params = [] )
Modifies the definition of an existing DataBrew dataset.
- UpdateProfileJob ( array $params = [] )
Modifies the definition of an existing profile job.
- UpdateProject ( array $params = [] )
Modifies the definition of an existing DataBrew project.
- UpdateRecipe ( array $params = [] )
Modifies the definition of the LATEST_WORKING version of a DataBrew recipe.
- UpdateRecipeJob ( array $params = [] )
Modifies the definition of an existing DataBrew recipe job.
- UpdateSchedule ( array $params = [] )
Modifies the definition of an existing DataBrew schedule.
Paginators
Paginators handle automatically iterating over paginated API results. Paginators are associated with specific API operations, and they accept the parameters that the corresponding API operation accepts. You can get a paginator from a client class using getPaginator($paginatorName, $operationParameters). This client supports the following paginators:
Operations
BatchDeleteRecipeVersion
$result = $client->batchDeleteRecipeVersion([/* ... */]); $promise = $client->batchDeleteRecipeVersionAsync([/* ... */]);
Deletes one or more versions of a recipe at a time.
The entire request will be rejected if:
-
The recipe does not exist.
-
There is an invalid version identifier in the list of versions.
-
The version list is empty.
-
The version list size exceeds 50.
-
The version list contains duplicate entries.
The request will complete successfully, but with partial failures, if:
-
A version does not exist.
-
A version is being used by a job.
-
You specify
LATEST_WORKING, but it's being used by a project. -
The version fails to be deleted.
The LATEST_WORKING version will only be deleted if the recipe has no other versions. If you try to delete LATEST_WORKING while other versions exist (or if they can't be deleted), then LATEST_WORKING will be listed as partial failure in the response.
Parameter Syntax
$result = $client->batchDeleteRecipeVersion([
'Name' => '<string>', // REQUIRED
'RecipeVersions' => ['<string>', ...], // REQUIRED
]);
Parameter Details
Members
- Name
-
- Required: Yes
- Type: string
The name of the recipe whose versions are to be deleted.
- RecipeVersions
-
- Required: Yes
- Type: Array of strings
An array of version identifiers, for the recipe versions to be deleted. You can specify numeric versions (
X.Y) orLATEST_WORKING.LATEST_PUBLISHEDis not supported.
Result Syntax
[
'Errors' => [
[
'ErrorCode' => '<string>',
'ErrorMessage' => '<string>',
'RecipeVersion' => '<string>',
],
// ...
],
'Name' => '<string>',
]
Result Details
Members
- Errors
-
- Type: Array of RecipeVersionErrorDetail structures
Errors, if any, that occurred while attempting to delete the recipe versions.
- Name
-
- Required: Yes
- Type: string
The name of the recipe that was modified.
Errors
-
Updating or deleting a resource can cause an inconsistent state.
-
One or more resources can't be found.
-
The input parameters for this request failed validation.
CreateDataset
$result = $client->createDataset([/* ... */]); $promise = $client->createDatasetAsync([/* ... */]);
Creates a new DataBrew dataset.
Parameter Syntax
$result = $client->createDataset([
'Format' => 'CSV|JSON|PARQUET|EXCEL',
'FormatOptions' => [
'Csv' => [
'Delimiter' => '<string>',
'HeaderRow' => true || false,
],
'Excel' => [
'HeaderRow' => true || false,
'SheetIndexes' => [<integer>, ...],
'SheetNames' => ['<string>', ...],
],
'Json' => [
'MultiLine' => true || false,
],
],
'Input' => [ // REQUIRED
'DataCatalogInputDefinition' => [
'CatalogId' => '<string>',
'DatabaseName' => '<string>', // REQUIRED
'TableName' => '<string>', // REQUIRED
'TempDirectory' => [
'Bucket' => '<string>', // REQUIRED
'Key' => '<string>',
],
],
'DatabaseInputDefinition' => [
'DatabaseTableName' => '<string>', // REQUIRED
'GlueConnectionName' => '<string>', // REQUIRED
'TempDirectory' => [
'Bucket' => '<string>', // REQUIRED
'Key' => '<string>',
],
],
'S3InputDefinition' => [
'Bucket' => '<string>', // REQUIRED
'Key' => '<string>',
],
],
'Name' => '<string>', // REQUIRED
'PathOptions' => [
'FilesLimit' => [
'MaxFiles' => <integer>, // REQUIRED
'Order' => 'DESCENDING|ASCENDING',
'OrderedBy' => 'LAST_MODIFIED_DATE',
],
'LastModifiedDateCondition' => [
'Expression' => '<string>', // REQUIRED
'ValuesMap' => ['<string>', ...], // REQUIRED
],
'Parameters' => [
'<PathParameterName>' => [
'CreateColumn' => true || false,
'DatetimeOptions' => [
'Format' => '<string>', // REQUIRED
'LocaleCode' => '<string>',
'TimezoneOffset' => '<string>',
],
'Filter' => [
'Expression' => '<string>', // REQUIRED
'ValuesMap' => ['<string>', ...], // REQUIRED
],
'Name' => '<string>', // REQUIRED
'Type' => 'Datetime|Number|String', // REQUIRED
],
// ...
],
],
'Tags' => ['<string>', ...],
]);
Parameter Details
Members
- Format
-
- Type: string
The file format of a dataset that is created from an S3 file or folder.
- FormatOptions
-
- Type: FormatOptions structure
Represents a set of options that define the structure of either comma-separated value (CSV), Excel, or JSON input.
- Input
-
- Required: Yes
- Type: Input structure
Represents information on how DataBrew can find data, in either the AWS Glue Data Catalog or Amazon S3.
- Name
-
- Required: Yes
- Type: string
The name of the dataset to be created. Valid characters are alphanumeric (A-Z, a-z, 0-9), hyphen (-), period (.), and space.
- PathOptions
-
- Type: PathOptions structure
A set of options that defines how DataBrew interprets an S3 path of the dataset.
- Tags
-
- Type: Associative array of custom strings keys (TagKey) to strings
Metadata tags to apply to this dataset.
Result Syntax
[
'Name' => '<string>',
]
Result Details
Errors
-
Access to the specified resource was denied.
-
Updating or deleting a resource can cause an inconsistent state.
-
ServiceQuotaExceededException:
A service quota is exceeded.
-
The input parameters for this request failed validation.
CreateProfileJob
$result = $client->createProfileJob([/* ... */]); $promise = $client->createProfileJobAsync([/* ... */]);
Creates a new job to analyze a dataset and create its data profile.
Parameter Syntax
$result = $client->createProfileJob([
'DatasetName' => '<string>', // REQUIRED
'EncryptionKeyArn' => '<string>',
'EncryptionMode' => 'SSE-KMS|SSE-S3',
'JobSample' => [
'Mode' => 'FULL_DATASET|CUSTOM_ROWS',
'Size' => <integer>,
],
'LogSubscription' => 'ENABLE|DISABLE',
'MaxCapacity' => <integer>,
'MaxRetries' => <integer>,
'Name' => '<string>', // REQUIRED
'OutputLocation' => [ // REQUIRED
'Bucket' => '<string>', // REQUIRED
'Key' => '<string>',
],
'RoleArn' => '<string>', // REQUIRED
'Tags' => ['<string>', ...],
'Timeout' => <integer>,
]);
Parameter Details
Members
- DatasetName
-
- Required: Yes
- Type: string
The name of the dataset that this job is to act upon.
- EncryptionKeyArn
-
- Type: string
The Amazon Resource Name (ARN) of an encryption key that is used to protect the job.
- EncryptionMode
-
- Type: string
The encryption mode for the job, which can be one of the following:
-
SSE-KMS-SSE-KMS- Server-side encryption with AWS KMS-managed keys. -
SSE-S3- Server-side encryption with keys managed by Amazon S3.
- JobSample
-
- Type: JobSample structure
Sample configuration for profile jobs only. Determines the number of rows on which the profile job will be executed. If a JobSample value is not provided, the default value will be used. The default value is CUSTOM_ROWS for the mode parameter and 20000 for the size parameter.
- LogSubscription
-
- Type: string
Enables or disables Amazon CloudWatch logging for the job. If logging is enabled, CloudWatch writes one log stream for each job run.
- MaxCapacity
-
- Type: int
The maximum number of nodes that DataBrew can use when the job processes data.
- MaxRetries
-
- Type: int
The maximum number of times to retry the job after a job run fails.
- Name
-
- Required: Yes
- Type: string
The name of the job to be created. Valid characters are alphanumeric (A-Z, a-z, 0-9), hyphen (-), period (.), and space.
- OutputLocation
-
- Required: Yes
- Type: S3Location structure
Represents an Amazon S3 location (bucket name and object key) where DataBrew can read input data, or write output from a job.
- RoleArn
-
- Required: Yes
- Type: string
The Amazon Resource Name (ARN) of the AWS Identity and Access Management (IAM) role to be assumed when DataBrew runs the job.
- Tags
-
- Type: Associative array of custom strings keys (TagKey) to strings
Metadata tags to apply to this job.
- Timeout
-
- Type: int
The job's timeout in minutes. A job that attempts to run longer than this timeout period ends with a status of
TIMEOUT.
Result Syntax
[
'Name' => '<string>',
]
Result Details
Errors
-
Access to the specified resource was denied.
-
Updating or deleting a resource can cause an inconsistent state.
-
One or more resources can't be found.
-
ServiceQuotaExceededException:
A service quota is exceeded.
-
The input parameters for this request failed validation.
CreateProject
$result = $client->createProject([/* ... */]); $promise = $client->createProjectAsync([/* ... */]);
Creates a new DataBrew project.
Parameter Syntax
$result = $client->createProject([
'DatasetName' => '<string>', // REQUIRED
'Name' => '<string>', // REQUIRED
'RecipeName' => '<string>', // REQUIRED
'RoleArn' => '<string>', // REQUIRED
'Sample' => [
'Size' => <integer>,
'Type' => 'FIRST_N|LAST_N|RANDOM', // REQUIRED
],
'Tags' => ['<string>', ...],
]);
Parameter Details
Members
- DatasetName
-
- Required: Yes
- Type: string
The name of an existing dataset to associate this project with.
- Name
-
- Required: Yes
- Type: string
A unique name for the new project. Valid characters are alphanumeric (A-Z, a-z, 0-9), hyphen (-), period (.), and space.
- RecipeName
-
- Required: Yes
- Type: string
The name of an existing recipe to associate with the project.
- RoleArn
-
- Required: Yes
- Type: string
The Amazon Resource Name (ARN) of the AWS Identity and Access Management (IAM) role to be assumed for this request.
- Sample
-
- Type: Sample structure
Represents the sample size and sampling type for DataBrew to use for interactive data analysis.
- Tags
-
- Type: Associative array of custom strings keys (TagKey) to strings
Metadata tags to apply to this project.
Result Syntax
[
'Name' => '<string>',
]
Result Details
Errors
-
Updating or deleting a resource can cause an inconsistent state.
-
An internal service failure occurred.
-
ServiceQuotaExceededException:
A service quota is exceeded.
-
The input parameters for this request failed validation.
CreateRecipe
$result = $client->createRecipe([/* ... */]); $promise = $client->createRecipeAsync([/* ... */]);
Creates a new DataBrew recipe.
Parameter Syntax
$result = $client->createRecipe([
'Description' => '<string>',
'Name' => '<string>', // REQUIRED
'Steps' => [ // REQUIRED
[
'Action' => [ // REQUIRED
'Operation' => '<string>', // REQUIRED
'Parameters' => ['<string>', ...],
],
'ConditionExpressions' => [
[
'Condition' => '<string>', // REQUIRED
'TargetColumn' => '<string>', // REQUIRED
'Value' => '<string>',
],
// ...
],
],
// ...
],
'Tags' => ['<string>', ...],
]);
Parameter Details
Members
- Description
-
- Type: string
A description for the recipe.
- Name
-
- Required: Yes
- Type: string
A unique name for the recipe. Valid characters are alphanumeric (A-Z, a-z, 0-9), hyphen (-), period (.), and space.
- Steps
-
- Required: Yes
- Type: Array of RecipeStep structures
An array containing the steps to be performed by the recipe. Each recipe step consists of one recipe action and (optionally) an array of condition expressions.
- Tags
-
- Type: Associative array of custom strings keys (TagKey) to strings
Metadata tags to apply to this recipe.
Result Syntax
[
'Name' => '<string>',
]
Result Details
Errors
-
Updating or deleting a resource can cause an inconsistent state.
-
ServiceQuotaExceededException:
A service quota is exceeded.
-
The input parameters for this request failed validation.
CreateRecipeJob
$result = $client->createRecipeJob([/* ... */]); $promise = $client->createRecipeJobAsync([/* ... */]);
Creates a new job to transform input data, using steps defined in an existing AWS Glue DataBrew recipe
Parameter Syntax
$result = $client->createRecipeJob([
'DatasetName' => '<string>',
'EncryptionKeyArn' => '<string>',
'EncryptionMode' => 'SSE-KMS|SSE-S3',
'LogSubscription' => 'ENABLE|DISABLE',
'MaxCapacity' => <integer>,
'MaxRetries' => <integer>,
'Name' => '<string>', // REQUIRED
'Outputs' => [ // REQUIRED
[
'CompressionFormat' => 'GZIP|LZ4|SNAPPY|BZIP2|DEFLATE|LZO|BROTLI|ZSTD|ZLIB',
'Format' => 'CSV|JSON|PARQUET|GLUEPARQUET|AVRO|ORC|XML',
'FormatOptions' => [
'Csv' => [
'Delimiter' => '<string>',
],
],
'Location' => [ // REQUIRED
'Bucket' => '<string>', // REQUIRED
'Key' => '<string>',
],
'Overwrite' => true || false,
'PartitionColumns' => ['<string>', ...],
],
// ...
],
'ProjectName' => '<string>',
'RecipeReference' => [
'Name' => '<string>', // REQUIRED
'RecipeVersion' => '<string>',
],
'RoleArn' => '<string>', // REQUIRED
'Tags' => ['<string>', ...],
'Timeout' => <integer>,
]);
Parameter Details
Members
- DatasetName
-
- Type: string
The name of the dataset that this job processes.
- EncryptionKeyArn
-
- Type: string
The Amazon Resource Name (ARN) of an encryption key that is used to protect the job.
- EncryptionMode
-
- Type: string
The encryption mode for the job, which can be one of the following:
-
SSE-KMS- Server-side encryption with keys managed by AWS KMS. -
SSE-S3- Server-side encryption with keys managed by Amazon S3.
- LogSubscription
-
- Type: string
Enables or disables Amazon CloudWatch logging for the job. If logging is enabled, CloudWatch writes one log stream for each job run.
- MaxCapacity
-
- Type: int
The maximum number of nodes that DataBrew can consume when the job processes data.
- MaxRetries
-
- Type: int
The maximum number of times to retry the job after a job run fails.
- Name
-
- Required: Yes
- Type: string
A unique name for the job. Valid characters are alphanumeric (A-Z, a-z, 0-9), hyphen (-), period (.), and space.
- Outputs
-
- Required: Yes
- Type: Array of Output structures
One or more artifacts that represent the output from running the job.
- ProjectName
-
- Type: string
Either the name of an existing project, or a combination of a recipe and a dataset to associate with the recipe.
- RecipeReference
-
- Type: RecipeReference structure
Represents the name and version of a DataBrew recipe.
- RoleArn
-
- Required: Yes
- Type: string
The Amazon Resource Name (ARN) of the AWS Identity and Access Management (IAM) role to be assumed when DataBrew runs the job.
- Tags
-
- Type: Associative array of custom strings keys (TagKey) to strings
Metadata tags to apply to this job.
- Timeout
-
- Type: int
The job's timeout in minutes. A job that attempts to run longer than this timeout period ends with a status of
TIMEOUT.
Result Syntax
[
'Name' => '<string>',
]
Result Details
Errors
-
Access to the specified resource was denied.
-
Updating or deleting a resource can cause an inconsistent state.
-
One or more resources can't be found.
-
ServiceQuotaExceededException:
A service quota is exceeded.
-
The input parameters for this request failed validation.
CreateSchedule
$result = $client->createSchedule([/* ... */]); $promise = $client->createScheduleAsync([/* ... */]);
Creates a new schedule for one or more DataBrew jobs. Jobs can be run at a specific date and time, or at regular intervals.
Parameter Syntax
$result = $client->createSchedule([
'CronExpression' => '<string>', // REQUIRED
'JobNames' => ['<string>', ...],
'Name' => '<string>', // REQUIRED
'Tags' => ['<string>', ...],
]);
Parameter Details
Members
- CronExpression
-
- Required: Yes
- Type: string
The date or dates and time or times when the jobs are to be run. For more information, see Cron expressions in the AWS Glue DataBrew Developer Guide.
- JobNames
-
- Type: Array of strings
The name or names of one or more jobs to be run.
- Name
-
- Required: Yes
- Type: string
A unique name for the schedule. Valid characters are alphanumeric (A-Z, a-z, 0-9), hyphen (-), period (.), and space.
- Tags
-
- Type: Associative array of custom strings keys (TagKey) to strings
Metadata tags to apply to this schedule.
Result Syntax
[
'Name' => '<string>',
]
Result Details
Errors
-
Updating or deleting a resource can cause an inconsistent state.
-
ServiceQuotaExceededException:
A service quota is exceeded.
-
The input parameters for this request failed validation.
DeleteDataset
$result = $client->deleteDataset([/* ... */]); $promise = $client->deleteDatasetAsync([/* ... */]);
Deletes a dataset from DataBrew.
Parameter Syntax
$result = $client->deleteDataset([
'Name' => '<string>', // REQUIRED
]);
Parameter Details
Result Syntax
[
'Name' => '<string>',
]
Result Details
Errors
-
Updating or deleting a resource can cause an inconsistent state.
-
One or more resources can't be found.
-
The input parameters for this request failed validation.
DeleteJob
$result = $client->deleteJob([/* ... */]); $promise = $client->deleteJobAsync([/* ... */]);
Deletes the specified DataBrew job.
Parameter Syntax
$result = $client->deleteJob([
'Name' => '<string>', // REQUIRED
]);
Parameter Details
Result Syntax
[
'Name' => '<string>',
]
Result Details
Errors
-
Updating or deleting a resource can cause an inconsistent state.
-
One or more resources can't be found.
-
The input parameters for this request failed validation.
DeleteProject
$result = $client->deleteProject([/* ... */]); $promise = $client->deleteProjectAsync([/* ... */]);
Deletes an existing DataBrew project.
Parameter Syntax
$result = $client->deleteProject([
'Name' => '<string>', // REQUIRED
]);
Parameter Details
Result Syntax
[
'Name' => '<string>',
]
Result Details
Errors
-
Updating or deleting a resource can cause an inconsistent state.
-
One or more resources can't be found.
-
The input parameters for this request failed validation.
DeleteRecipeVersion
$result = $client->deleteRecipeVersion([/* ... */]); $promise = $client->deleteRecipeVersionAsync([/* ... */]);
Deletes a single version of a DataBrew recipe.
Parameter Syntax
$result = $client->deleteRecipeVersion([
'Name' => '<string>', // REQUIRED
'RecipeVersion' => '<string>', // REQUIRED
]);
Parameter Details
Members
Result Syntax
[
'Name' => '<string>',
'RecipeVersion' => '<string>',
]
Result Details
Members
Errors
-
Updating or deleting a resource can cause an inconsistent state.
-
One or more resources can't be found.
-
The input parameters for this request failed validation.
DeleteSchedule
$result = $client->deleteSchedule([/* ... */]); $promise = $client->deleteScheduleAsync([/* ... */]);
Deletes the specified DataBrew schedule.
Parameter Syntax
$result = $client->deleteSchedule([
'Name' => '<string>', // REQUIRED
]);
Parameter Details
Result Syntax
[
'Name' => '<string>',
]
Result Details
Errors
-
One or more resources can't be found.
-
The input parameters for this request failed validation.
DescribeDataset
$result = $client->describeDataset([/* ... */]); $promise = $client->describeDatasetAsync([/* ... */]);
Returns the definition of a specific DataBrew dataset.
Parameter Syntax
$result = $client->describeDataset([
'Name' => '<string>', // REQUIRED
]);
Parameter Details
Result Syntax
[
'CreateDate' => <DateTime>,
'CreatedBy' => '<string>',
'Format' => 'CSV|JSON|PARQUET|EXCEL',
'FormatOptions' => [
'Csv' => [
'Delimiter' => '<string>',
'HeaderRow' => true || false,
],
'Excel' => [
'HeaderRow' => true || false,
'SheetIndexes' => [<integer>, ...],
'SheetNames' => ['<string>', ...],
],
'Json' => [
'MultiLine' => true || false,
],
],
'Input' => [
'DataCatalogInputDefinition' => [
'CatalogId' => '<string>',
'DatabaseName' => '<string>',
'TableName' => '<string>',
'TempDirectory' => [
'Bucket' => '<string>',
'Key' => '<string>',
],
],
'DatabaseInputDefinition' => [
'DatabaseTableName' => '<string>',
'GlueConnectionName' => '<string>',
'TempDirectory' => [
'Bucket' => '<string>',
'Key' => '<string>',
],
],
'S3InputDefinition' => [
'Bucket' => '<string>',
'Key' => '<string>',
],
],
'LastModifiedBy' => '<string>',
'LastModifiedDate' => <DateTime>,
'Name' => '<string>',
'PathOptions' => [
'FilesLimit' => [
'MaxFiles' => <integer>,
'Order' => 'DESCENDING|ASCENDING',
'OrderedBy' => 'LAST_MODIFIED_DATE',
],
'LastModifiedDateCondition' => [
'Expression' => '<string>',
'ValuesMap' => ['<string>', ...],
],
'Parameters' => [
'<PathParameterName>' => [
'CreateColumn' => true || false,
'DatetimeOptions' => [
'Format' => '<string>',
'LocaleCode' => '<string>',
'TimezoneOffset' => '<string>',
],
'Filter' => [
'Expression' => '<string>',
'ValuesMap' => ['<string>', ...],
],
'Name' => '<string>',
'Type' => 'Datetime|Number|String',
],
// ...
],
],
'ResourceArn' => '<string>',
'Source' => 'S3|DATA-CATALOG|DATABASE',
'Tags' => ['<string>', ...],
]
Result Details
Members
- CreateDate
-
- Type: timestamp (string|DateTime or anything parsable by strtotime)
The date and time that the dataset was created.
- CreatedBy
-
- Type: string
The identifier (user name) of the user who created the dataset.
- Format
-
- Type: string
The file format of a dataset that is created from an S3 file or folder.
- FormatOptions
-
- Type: FormatOptions structure
Represents a set of options that define the structure of either comma-separated value (CSV), Excel, or JSON input.
- Input
-
- Required: Yes
- Type: Input structure
Represents information on how DataBrew can find data, in either the AWS Glue Data Catalog or Amazon S3.
- LastModifiedBy
-
- Type: string
The identifier (user name) of the user who last modified the dataset.
- LastModifiedDate
-
- Type: timestamp (string|DateTime or anything parsable by strtotime)
The date and time that the dataset was last modified.
- Name
-
- Required: Yes
- Type: string
The name of the dataset.
- PathOptions
-
- Type: PathOptions structure
A set of options that defines how DataBrew interprets an S3 path of the dataset.
- ResourceArn
-
- Type: string
The Amazon Resource Name (ARN) of the dataset.
- Source
-
- Type: string
The location of the data for this dataset, Amazon S3 or the AWS Glue Data Catalog.
- Tags
-
- Type: Associative array of custom strings keys (TagKey) to strings
Metadata tags associated with this dataset.
Errors
-
One or more resources can't be found.
-
The input parameters for this request failed validation.
DescribeJob
$result = $client->describeJob([/* ... */]); $promise = $client->describeJobAsync([/* ... */]);
Returns the definition of a specific DataBrew job.
Parameter Syntax
$result = $client->describeJob([
'Name' => '<string>', // REQUIRED
]);
Parameter Details
Result Syntax
[
'CreateDate' => <DateTime>,
'CreatedBy' => '<string>',
'DatasetName' => '<string>',
'EncryptionKeyArn' => '<string>',
'EncryptionMode' => 'SSE-KMS|SSE-S3',
'JobSample' => [
'Mode' => 'FULL_DATASET|CUSTOM_ROWS',
'Size' => <integer>,
],
'LastModifiedBy' => '<string>',
'LastModifiedDate' => <DateTime>,
'LogSubscription' => 'ENABLE|DISABLE',
'MaxCapacity' => <integer>,
'MaxRetries' => <integer>,
'Name' => '<string>',
'Outputs' => [
[
'CompressionFormat' => 'GZIP|LZ4|SNAPPY|BZIP2|DEFLATE|LZO|BROTLI|ZSTD|ZLIB',
'Format' => 'CSV|JSON|PARQUET|GLUEPARQUET|AVRO|ORC|XML',
'FormatOptions' => [
'Csv' => [
'Delimiter' => '<string>',
],
],
'Location' => [
'Bucket' => '<string>',
'Key' => '<string>',
],
'Overwrite' => true || false,
'PartitionColumns' => ['<string>', ...],
],
// ...
],
'ProjectName' => '<string>',
'RecipeReference' => [
'Name' => '<string>',
'RecipeVersion' => '<string>',
],
'ResourceArn' => '<string>',
'RoleArn' => '<string>',
'Tags' => ['<string>', ...],
'Timeout' => <integer>,
'Type' => 'PROFILE|RECIPE',
]
Result Details
Members
- CreateDate
-
- Type: timestamp (string|DateTime or anything parsable by strtotime)
The date and time that the job was created.
- CreatedBy
-
- Type: string
The identifier (user name) of the user associated with the creation of the job.
- DatasetName
-
- Type: string
The dataset that the job acts upon.
- EncryptionKeyArn
-
- Type: string
The Amazon Resource Name (ARN) of an encryption key that is used to protect the job.
- EncryptionMode
-
- Type: string
The encryption mode for the job, which can be one of the following:
-
SSE-KMS- Server-side encryption with keys managed by AWS KMS. -
SSE-S3- Server-side encryption with keys managed by Amazon S3.
- JobSample
-
- Type: JobSample structure
Sample configuration for profile jobs only. Determines the number of rows on which the profile job will be executed.
- LastModifiedBy
-
- Type: string
The identifier (user name) of the user who last modified the job.
- LastModifiedDate
-
- Type: timestamp (string|DateTime or anything parsable by strtotime)
The date and time that the job was last modified.
- LogSubscription
-
- Type: string
Indicates whether Amazon CloudWatch logging is enabled for this job.
- MaxCapacity
-
- Type: int
The maximum number of compute nodes that DataBrew can consume when the job processes data.
- MaxRetries
-
- Type: int
The maximum number of times to retry the job after a job run fails.
- Name
-
- Required: Yes
- Type: string
The name of the job.
- Outputs
-
- Type: Array of Output structures
One or more artifacts that represent the output from running the job.
- ProjectName
-
- Type: string
The DataBrew project associated with this job.
- RecipeReference
-
- Type: RecipeReference structure
Represents the name and version of a DataBrew recipe.
- ResourceArn
-
- Type: string
The Amazon Resource Name (ARN) of the job.
- RoleArn
-
- Type: string
The ARN of the AWS Identity and Access Management (IAM) role to be assumed when DataBrew runs the job.
- Tags
-
- Type: Associative array of custom strings keys (TagKey) to strings
Metadata tags associated with this job.
- Timeout
-
- Type: int
The job's timeout in minutes. A job that attempts to run longer than this timeout period ends with a status of
TIMEOUT. - Type
-
- Type: string
The job type, which must be one of the following:
-
PROFILE- The job analyzes the dataset to determine its size, data types, data distribution, and more. -
RECIPE- The job applies one or more transformations to a dataset.
Errors
-
One or more resources can't be found.
-
The input parameters for this request failed validation.
DescribeJobRun
$result = $client->describeJobRun([/* ... */]); $promise = $client->describeJobRunAsync([/* ... */]);
Represents one run of a DataBrew job.
Parameter Syntax
$result = $client->describeJobRun([
'Name' => '<string>', // REQUIRED
'RunId' => '<string>', // REQUIRED
]);
Parameter Details
Members
Result Syntax
[
'Attempt' => <integer>,
'CompletedOn' => <DateTime>,
'DatasetName' => '<string>',
'ErrorMessage' => '<string>',
'ExecutionTime' => <integer>,
'JobName' => '<string>',
'JobSample' => [
'Mode' => 'FULL_DATASET|CUSTOM_ROWS',
'Size' => <integer>,
],
'LogGroupName' => '<string>',
'LogSubscription' => 'ENABLE|DISABLE',
'Outputs' => [
[
'CompressionFormat' => 'GZIP|LZ4|SNAPPY|BZIP2|DEFLATE|LZO|BROTLI|ZSTD|ZLIB',
'Format' => 'CSV|JSON|PARQUET|GLUEPARQUET|AVRO|ORC|XML',
'FormatOptions' => [
'Csv' => [
'Delimiter' => '<string>',
],
],
'Location' => [
'Bucket' => '<string>',
'Key' => '<string>',
],
'Overwrite' => true || false,
'PartitionColumns' => ['<string>', ...],
],
// ...
],
'RecipeReference' => [
'Name' => '<string>',
'RecipeVersion' => '<string>',
],
'RunId' => '<string>',
'StartedBy' => '<string>',
'StartedOn' => <DateTime>,
'State' => 'STARTING|RUNNING|STOPPING|STOPPED|SUCCEEDED|FAILED|TIMEOUT',
]
Result Details
Members
- Attempt
-
- Type: int
The number of times that DataBrew has attempted to run the job.
- CompletedOn
-
- Type: timestamp (string|DateTime or anything parsable by strtotime)
The date and time when the job completed processing.
- DatasetName
-
- Type: string
The name of the dataset for the job to process.
- ErrorMessage
-
- Type: string
A message indicating an error (if any) that was encountered when the job ran.
- ExecutionTime
-
- Type: int
The amount of time, in seconds, during which the job run consumed resources.
- JobName
-
- Required: Yes
- Type: string
The name of the job being processed during this run.
- JobSample
-
- Type: JobSample structure
Sample configuration for profile jobs only. Determines the number of rows on which the profile job will be executed. If a JobSample value is not provided, the default value will be used. The default value is CUSTOM_ROWS for the mode parameter and 20000 for the size parameter.
- LogGroupName
-
- Type: string
The name of an Amazon CloudWatch log group, where the job writes diagnostic messages when it runs.
- LogSubscription
-
- Type: string
The current status of Amazon CloudWatch logging for the job run.
- Outputs
-
- Type: Array of Output structures
One or more output artifacts from a job run.
- RecipeReference
-
- Type: RecipeReference structure
Represents the name and version of a DataBrew recipe.
- RunId
-
- Type: string
The unique identifier of the job run.
- StartedBy
-
- Type: string
The Amazon Resource Name (ARN) of the user who started the job run.
- StartedOn
-
- Type: timestamp (string|DateTime or anything parsable by strtotime)
The date and time when the job run began.
- State
-
- Type: string
The current state of the job run entity itself.
Errors
-
One or more resources can't be found.
-
The input parameters for this request failed validation.
DescribeProject
$result = $client->describeProject([/* ... */]); $promise = $client->describeProjectAsync([/* ... */]);
Returns the definition of a specific DataBrew project.
Parameter Syntax
$result = $client->describeProject([
'Name' => '<string>', // REQUIRED
]);
Parameter Details
Result Syntax
[
'CreateDate' => <DateTime>,
'CreatedBy' => '<string>',
'DatasetName' => '<string>',
'LastModifiedBy' => '<string>',
'LastModifiedDate' => <DateTime>,
'Name' => '<string>',
'OpenDate' => <DateTime>,
'OpenedBy' => '<string>',
'RecipeName' => '<string>',
'ResourceArn' => '<string>',
'RoleArn' => '<string>',
'Sample' => [
'Size' => <integer>,
'Type' => 'FIRST_N|LAST_N|RANDOM',
],
'SessionStatus' => 'ASSIGNED|FAILED|INITIALIZING|PROVISIONING|READY|RECYCLING|ROTATING|TERMINATED|TERMINATING|UPDATING',
'Tags' => ['<string>', ...],
]
Result Details
Members
- CreateDate
-
- Type: timestamp (string|DateTime or anything parsable by strtotime)
The date and time that the project was created.
- CreatedBy
-
- Type: string
The identifier (user name) of the user who created the project.
- DatasetName
-
- Type: string
The dataset associated with the project.
- LastModifiedBy
-
- Type: string
The identifier (user name) of the user who last modified the project.
- LastModifiedDate
-
- Type: timestamp (string|DateTime or anything parsable by strtotime)
The date and time that the project was last modified.
- Name
-
- Required: Yes
- Type: string
The name of the project.
- OpenDate
-
- Type: timestamp (string|DateTime or anything parsable by strtotime)
The date and time when the project was opened.
- OpenedBy
-
- Type: string
The identifier (user name) of the user that opened the project for use.
- RecipeName
-
- Type: string
The recipe associated with this job.
- ResourceArn
-
- Type: string
The Amazon Resource Name (ARN) of the project.
- RoleArn
-
- Type: string
The ARN of the AWS Identity and Access Management (IAM) role to be assumed when DataBrew runs the job.
- Sample
-
- Type: Sample structure
Represents the sample size and sampling type for DataBrew to use for interactive data analysis.
- SessionStatus
-
- Type: string
Describes the current state of the session:
-
PROVISIONING- allocating resources for the session. -
INITIALIZING- getting the session ready for first use. -
ASSIGNED- the session is ready for use.
- Tags
-
- Type: Associative array of custom strings keys (TagKey) to strings
Metadata tags associated with this project.
Errors
-
One or more resources can't be found.
-
The input parameters for this request failed validation.
DescribeRecipe
$result = $client->describeRecipe([/* ... */]); $promise = $client->describeRecipeAsync([/* ... */]);
Returns the definition of a specific DataBrew recipe corresponding to a particular version.
Parameter Syntax
$result = $client->describeRecipe([
'Name' => '<string>', // REQUIRED
'RecipeVersion' => '<string>',
]);
Parameter Details
Members
Result Syntax
[
'CreateDate' => <DateTime>,
'CreatedBy' => '<string>',
'Description' => '<string>',
'LastModifiedBy' => '<string>',
'LastModifiedDate' => <DateTime>,
'Name' => '<string>',
'ProjectName' => '<string>',
'PublishedBy' => '<string>',
'PublishedDate' => <DateTime>,
'RecipeVersion' => '<string>',
'ResourceArn' => '<string>',
'Steps' => [
[
'Action' => [
'Operation' => '<string>',
'Parameters' => ['<string>', ...],
],
'ConditionExpressions' => [
[
'Condition' => '<string>',
'TargetColumn' => '<string>',
'Value' => '<string>',
],
// ...
],
],
// ...
],
'Tags' => ['<string>', ...],
]
Result Details
Members
- CreateDate
-
- Type: timestamp (string|DateTime or anything parsable by strtotime)
The date and time that the recipe was created.
- CreatedBy
-
- Type: string
The identifier (user name) of the user who created the recipe.
- Description
-
- Type: string
The description of the recipe.
- LastModifiedBy
-
- Type: string
The identifier (user name) of the user who last modified the recipe.
- LastModifiedDate
-
- Type: timestamp (string|DateTime or anything parsable by strtotime)
The date and time that the recipe was last modified.
- Name
-
- Required: Yes
- Type: string
The name of the recipe.
- ProjectName
-
- Type: string
The name of the project associated with this recipe.
- PublishedBy
-
- Type: string
The identifier (user name) of the user who last published the recipe.
- PublishedDate
-
- Type: timestamp (string|DateTime or anything parsable by strtotime)
The date and time when the recipe was last published.
- RecipeVersion
-
- Type: string
The recipe version identifier.
- ResourceArn
-
- Type: string
The ARN of the recipe.
- Steps
-
- Type: Array of RecipeStep structures
One or more steps to be performed by the recipe. Each step consists of an action, and the conditions under which the action should succeed.
- Tags
-
- Type: Associative array of custom strings keys (TagKey) to strings
Metadata tags associated with this project.
Errors
-
One or more resources can't be found.
-
The input parameters for this request failed validation.
DescribeSchedule
$result = $client->describeSchedule([/* ... */]); $promise = $client->describeScheduleAsync([/* ... */]);
Returns the definition of a specific DataBrew schedule.
Parameter Syntax
$result = $client->describeSchedule([
'Name' => '<string>', // REQUIRED
]);
Parameter Details
Result Syntax
[
'CreateDate' => <DateTime>,
'CreatedBy' => '<string>',
'CronExpression' => '<string>',
'JobNames' => ['<string>', ...],
'LastModifiedBy' => '<string>',
'LastModifiedDate' => <DateTime>,
'Name' => '<string>',
'ResourceArn' => '<string>',
'Tags' => ['<string>', ...],
]
Result Details
Members
- CreateDate
-
- Type: timestamp (string|DateTime or anything parsable by strtotime)
The date and time that the schedule was created.
- CreatedBy
-
- Type: string
The identifier (user name) of the user who created the schedule.
- CronExpression
-
- Type: string
The date or dates and time or times when the jobs are to be run for the schedule. For more information, see Cron expressions in the AWS Glue DataBrew Developer Guide.
- JobNames
-
- Type: Array of strings
The name or names of one or more jobs to be run by using the schedule.
- LastModifiedBy
-
- Type: string
The identifier (user name) of the user who last modified the schedule.
- LastModifiedDate
-
- Type: timestamp (string|DateTime or anything parsable by strtotime)
The date and time that the schedule was last modified.
- Name
-
- Required: Yes
- Type: string
The name of the schedule.
- ResourceArn
-
- Type: string
The Amazon Resource Name (ARN) of the schedule.
- Tags
-
- Type: Associative array of custom strings keys (TagKey) to strings
Metadata tags associated with this schedule.
Errors
-
One or more resources can't be found.
-
The input parameters for this request failed validation.
ListDatasets
$result = $client->listDatasets([/* ... */]); $promise = $client->listDatasetsAsync([/* ... */]);
Lists all of the DataBrew datasets.
Parameter Syntax
$result = $client->listDatasets([
'MaxResults' => <integer>,
'NextToken' => '<string>',
]);
Parameter Details
Members
Result Syntax
[
'Datasets' => [
[
'AccountId' => '<string>',
'CreateDate' => <DateTime>,
'CreatedBy' => '<string>',
'Format' => 'CSV|JSON|PARQUET|EXCEL',
'FormatOptions' => [
'Csv' => [
'Delimiter' => '<string>',
'HeaderRow' => true || false,
],
'Excel' => [
'HeaderRow' => true || false,
'SheetIndexes' => [<integer>, ...],
'SheetNames' => ['<string>', ...],
],
'Json' => [
'MultiLine' => true || false,
],
],
'Input' => [
'DataCatalogInputDefinition' => [
'CatalogId' => '<string>',
'DatabaseName' => '<string>',
'TableName' => '<string>',
'TempDirectory' => [
'Bucket' => '<string>',
'Key' => '<string>',
],
],
'DatabaseInputDefinition' => [
'DatabaseTableName' => '<string>',
'GlueConnectionName' => '<string>',
'TempDirectory' => [
'Bucket' => '<string>',
'Key' => '<string>',
],
],
'S3InputDefinition' => [
'Bucket' => '<string>',
'Key' => '<string>',
],
],
'LastModifiedBy' => '<string>',
'LastModifiedDate' => <DateTime>,
'Name' => '<string>',
'PathOptions' => [
'FilesLimit' => [
'MaxFiles' => <integer>,
'Order' => 'DESCENDING|ASCENDING',
'OrderedBy' => 'LAST_MODIFIED_DATE',
],
'LastModifiedDateCondition' => [
'Expression' => '<string>',
'ValuesMap' => ['<string>', ...],
],
'Parameters' => [
'<PathParameterName>' => [
'CreateColumn' => true || false,
'DatetimeOptions' => [
'Format' => '<string>',
'LocaleCode' => '<string>',
'TimezoneOffset' => '<string>',
],
'Filter' => [
'Expression' => '<string>',
'ValuesMap' => ['<string>', ...],
],
'Name' => '<string>',
'Type' => 'Datetime|Number|String',
],
// ...
],
],
'ResourceArn' => '<string>',
'Source' => 'S3|DATA-CATALOG|DATABASE',
'Tags' => ['<string>', ...],
],
// ...
],
'NextToken' => '<string>',
]
Result Details
Members
- Datasets
-
- Required: Yes
- Type: Array of Dataset structures
A list of datasets that are defined.
- NextToken
-
- Type: string
A token that you can use in a subsequent call to retrieve the next set of results.
Errors
-
The input parameters for this request failed validation.
ListJobRuns
$result = $client->listJobRuns([/* ... */]); $promise = $client->listJobRunsAsync([/* ... */]);
Lists all of the previous runs of a particular DataBrew job.
Parameter Syntax
$result = $client->listJobRuns([
'MaxResults' => <integer>,
'Name' => '<string>', // REQUIRED
'NextToken' => '<string>',
]);
Parameter Details
Members
Result Syntax
[
'JobRuns' => [
[
'Attempt' => <integer>,
'CompletedOn' => <DateTime>,
'DatasetName' => '<string>',
'ErrorMessage' => '<string>',
'ExecutionTime' => <integer>,
'JobName' => '<string>',
'JobSample' => [
'Mode' => 'FULL_DATASET|CUSTOM_ROWS',
'Size' => <integer>,
],
'LogGroupName' => '<string>',
'LogSubscription' => 'ENABLE|DISABLE',
'Outputs' => [
[
'CompressionFormat' => 'GZIP|LZ4|SNAPPY|BZIP2|DEFLATE|LZO|BROTLI|ZSTD|ZLIB',
'Format' => 'CSV|JSON|PARQUET|GLUEPARQUET|AVRO|ORC|XML',
'FormatOptions' => [
'Csv' => [
'Delimiter' => '<string>',
],
],
'Location' => [
'Bucket' => '<string>',
'Key' => '<string>',
],
'Overwrite' => true || false,
'PartitionColumns' => ['<string>', ...],
],
// ...
],
'RecipeReference' => [
'Name' => '<string>',
'RecipeVersion' => '<string>',
],
'RunId' => '<string>',
'StartedBy' => '<string>',
'StartedOn' => <DateTime>,
'State' => 'STARTING|RUNNING|STOPPING|STOPPED|SUCCEEDED|FAILED|TIMEOUT',
],
// ...
],
'NextToken' => '<string>',
]
Result Details
Members
- JobRuns
-
- Required: Yes
- Type: Array of JobRun structures
A list of job runs that have occurred for the specified job.
- NextToken
-
- Type: string
A token that you can use in a subsequent call to retrieve the next set of results.
Errors
-
One or more resources can't be found.
-
The input parameters for this request failed validation.
ListJobs
$result = $client->listJobs([/* ... */]); $promise = $client->listJobsAsync([/* ... */]);
Lists all of the DataBrew jobs that are defined.
Parameter Syntax
$result = $client->listJobs([
'DatasetName' => '<string>',
'MaxResults' => <integer>,
'NextToken' => '<string>',
'ProjectName' => '<string>',
]);
Parameter Details
Members
- DatasetName
-
- Type: string
The name of a dataset. Using this parameter indicates to return only those jobs that act on the specified dataset.
- MaxResults
-
- Type: int
The maximum number of results to return in this request.
- NextToken
-
- Type: string
A token generated by DataBrew that specifies where to continue pagination if a previous request was truncated. To get the next set of pages, pass in the NextToken value from the response object of the previous page call.
- ProjectName
-
- Type: string
The name of a project. Using this parameter indicates to return only those jobs that are associated with the specified project.
Result Syntax
[
'Jobs' => [
[
'AccountId' => '<string>',
'CreateDate' => <DateTime>,
'CreatedBy' => '<string>',
'DatasetName' => '<string>',
'EncryptionKeyArn' => '<string>',
'EncryptionMode' => 'SSE-KMS|SSE-S3',
'JobSample' => [
'Mode' => 'FULL_DATASET|CUSTOM_ROWS',
'Size' => <integer>,
],
'LastModifiedBy' => '<string>',
'LastModifiedDate' => <DateTime>,
'LogSubscription' => 'ENABLE|DISABLE',
'MaxCapacity' => <integer>,
'MaxRetries' => <integer>,
'Name' => '<string>',
'Outputs' => [
[
'CompressionFormat' => 'GZIP|LZ4|SNAPPY|BZIP2|DEFLATE|LZO|BROTLI|ZSTD|ZLIB',
'Format' => 'CSV|JSON|PARQUET|GLUEPARQUET|AVRO|ORC|XML',
'FormatOptions' => [
'Csv' => [
'Delimiter' => '<string>',
],
],
'Location' => [
'Bucket' => '<string>',
'Key' => '<string>',
],
'Overwrite' => true || false,
'PartitionColumns' => ['<string>', ...],
],
// ...
],
'ProjectName' => '<string>',
'RecipeReference' => [
'Name' => '<string>',
'RecipeVersion' => '<string>',
],
'ResourceArn' => '<string>',
'RoleArn' => '<string>',
'Tags' => ['<string>', ...],
'Timeout' => <integer>,
'Type' => 'PROFILE|RECIPE',
],
// ...
],
'NextToken' => '<string>',
]
Result Details
Members
- Jobs
-
- Required: Yes
- Type: Array of Job structures
A list of jobs that are defined.
- NextToken
-
- Type: string
A token that you can use in a subsequent call to retrieve the next set of results.
Errors
-
The input parameters for this request failed validation.
ListProjects
$result = $client->listProjects([/* ... */]); $promise = $client->listProjectsAsync([/* ... */]);
Lists all of the DataBrew projects that are defined.
Parameter Syntax
$result = $client->listProjects([
'MaxResults' => <integer>,
'NextToken' => '<string>',
]);
Parameter Details
Members
Result Syntax
[
'NextToken' => '<string>',
'Projects' => [
[
'AccountId' => '<string>',
'CreateDate' => <DateTime>,
'CreatedBy' => '<string>',
'DatasetName' => '<string>',
'LastModifiedBy' => '<string>',
'LastModifiedDate' => <DateTime>,
'Name' => '<string>',
'OpenDate' => <DateTime>,
'OpenedBy' => '<string>',
'RecipeName' => '<string>',
'ResourceArn' => '<string>',
'RoleArn' => '<string>',
'Sample' => [
'Size' => <integer>,
'Type' => 'FIRST_N|LAST_N|RANDOM',
],
'Tags' => ['<string>', ...],
],
// ...
],
]
Result Details
Members
- NextToken
-
- Type: string
A token that you can use in a subsequent call to retrieve the next set of results.
- Projects
-
- Required: Yes
- Type: Array of Project structures
A list of projects that are defined .
Errors
-
The input parameters for this request failed validation.
ListRecipeVersions
$result = $client->listRecipeVersions([/* ... */]); $promise = $client->listRecipeVersionsAsync([/* ... */]);
Lists the versions of a particular DataBrew recipe, except for LATEST_WORKING.
Parameter Syntax
$result = $client->listRecipeVersions([
'MaxResults' => <integer>,
'Name' => '<string>', // REQUIRED
'NextToken' => '<string>',
]);
Parameter Details
Members
Result Syntax
[
'NextToken' => '<string>',
'Recipes' => [
[
'CreateDate' => <DateTime>,
'CreatedBy' => '<string>',
'Description' => '<string>',
'LastModifiedBy' => '<string>',
'LastModifiedDate' => <DateTime>,
'Name' => '<string>',
'ProjectName' => '<string>',
'PublishedBy' => '<string>',
'PublishedDate' => <DateTime>,
'RecipeVersion' => '<string>',
'ResourceArn' => '<string>',
'Steps' => [
[
'Action' => [
'Operation' => '<string>',
'Parameters' => ['<string>', ...],
],
'ConditionExpressions' => [
[
'Condition' => '<string>',
'TargetColumn' => '<string>',
'Value' => '<string>',
],
// ...
],
],
// ...
],
'Tags' => ['<string>', ...],
],
// ...
],
]
Result Details
Members
- NextToken
-
- Type: string
A token that you can use in a subsequent call to retrieve the next set of results.
- Recipes
-
- Required: Yes
- Type: Array of Recipe structures
A list of versions for the specified recipe.
Errors
-
The input parameters for this request failed validation.
ListRecipes
$result = $client->listRecipes([/* ... */]); $promise = $client->listRecipesAsync([/* ... */]);
Lists all of the DataBrew recipes that are defined.
Parameter Syntax
$result = $client->listRecipes([
'MaxResults' => <integer>,
'NextToken' => '<string>',
'RecipeVersion' => '<string>',
]);
Parameter Details
Members
- MaxResults
-
- Type: int
The maximum number of results to return in this request.
- NextToken
-
- Type: string
The token returned by a previous call to retrieve the next set of results.
- RecipeVersion
-
- Type: string
Return only those recipes with a version identifier of
LATEST_WORKINGorLATEST_PUBLISHED. IfRecipeVersionis omitted,ListRecipesreturns all of theLATEST_PUBLISHEDrecipe versions.Valid values:
LATEST_WORKING|LATEST_PUBLISHED
Result Syntax
[
'NextToken' => '<string>',
'Recipes' => [
[
'CreateDate' => <DateTime>,
'CreatedBy' => '<string>',
'Description' => '<string>',
'LastModifiedBy' => '<string>',
'LastModifiedDate' => <DateTime>,
'Name' => '<string>',
'ProjectName' => '<string>',
'PublishedBy' => '<string>',
'PublishedDate' => <DateTime>,
'RecipeVersion' => '<string>',
'ResourceArn' => '<string>',
'Steps' => [
[
'Action' => [
'Operation' => '<string>',
'Parameters' => ['<string>', ...],
],
'ConditionExpressions' => [
[
'Condition' => '<string>',
'TargetColumn' => '<string>',
'Value' => '<string>',
],
// ...
],
],
// ...
],
'Tags' => ['<string>', ...],
],
// ...
],
]
Result Details
Members
- NextToken
-
- Type: string
A token that you can use in a subsequent call to retrieve the next set of results.
- Recipes
-
- Required: Yes
- Type: Array of Recipe structures
A list of recipes that are defined.
Errors
-
The input parameters for this request failed validation.
ListSchedules
$result = $client->listSchedules([/* ... */]); $promise = $client->listSchedulesAsync([/* ... */]);
Lists the DataBrew schedules that are defined.
Parameter Syntax
$result = $client->listSchedules([
'JobName' => '<string>',
'MaxResults' => <integer>,
'NextToken' => '<string>',
]);
Parameter Details
Members
Result Syntax
[
'NextToken' => '<string>',
'Schedules' => [
[
'AccountId' => '<string>',
'CreateDate' => <DateTime>,
'CreatedBy' => '<string>',
'CronExpression' => '<string>',
'JobNames' => ['<string>', ...],
'LastModifiedBy' => '<string>',
'LastModifiedDate' => <DateTime>,
'Name' => '<string>',
'ResourceArn' => '<string>',
'Tags' => ['<string>', ...],
],
// ...
],
]
Result Details
Members
- NextToken
-
- Type: string
A token that you can use in a subsequent call to retrieve the next set of results.
- Schedules
-
- Required: Yes
- Type: Array of Schedule structures
A list of schedules that are defined.
Errors
-
The input parameters for this request failed validation.
ListTagsForResource
$result = $client->listTagsForResource([/* ... */]); $promise = $client->listTagsForResourceAsync([/* ... */]);
Lists all the tags for a DataBrew resource.
Parameter Syntax
$result = $client->listTagsForResource([
'ResourceArn' => '<string>', // REQUIRED
]);
Parameter Details
Members
Result Syntax
[
'Tags' => ['<string>', ...],
]
Result Details
Members
Errors
-
An internal service failure occurred.
-
One or more resources can't be found.
-
The input parameters for this request failed validation.
PublishRecipe
$result = $client->publishRecipe([/* ... */]); $promise = $client->publishRecipeAsync([/* ... */]);
Publishes a new version of a DataBrew recipe.
Parameter Syntax
$result = $client->publishRecipe([
'Description' => '<string>',
'Name' => '<string>', // REQUIRED
]);
Parameter Details
Members
Result Syntax
[
'Name' => '<string>',
]
Result Details
Errors
-
The input parameters for this request failed validation.
-
One or more resources can't be found.
-
ServiceQuotaExceededException:
A service quota is exceeded.
SendProjectSessionAction
$result = $client->sendProjectSessionAction([/* ... */]); $promise = $client->sendProjectSessionActionAsync([/* ... */]);
Performs a recipe step within an interactive DataBrew session that's currently open.
Parameter Syntax
$result = $client->sendProjectSessionAction([
'ClientSessionId' => '<string>',
'Name' => '<string>', // REQUIRED
'Preview' => true || false,
'RecipeStep' => [
'Action' => [ // REQUIRED
'Operation' => '<string>', // REQUIRED
'Parameters' => ['<string>', ...],
],
'ConditionExpressions' => [
[
'Condition' => '<string>', // REQUIRED
'TargetColumn' => '<string>', // REQUIRED
'Value' => '<string>',
],
// ...
],
],
'StepIndex' => <integer>,
'ViewFrame' => [
'ColumnRange' => <integer>,
'HiddenColumns' => ['<string>', ...],
'StartColumnIndex' => <integer>, // REQUIRED
],
]);
Parameter Details
Members
- ClientSessionId
-
- Type: string
A unique identifier for an interactive session that's currently open and ready for work. The action will be performed on this session.
- Name
-
- Required: Yes
- Type: string
The name of the project to apply the action to.
- Preview
-
- Type: boolean
If true, the result of the recipe step will be returned, but not applied.
- RecipeStep
-
- Type: RecipeStep structure
Represents a single step from a DataBrew recipe to be performed.
- StepIndex
-
- Type: int
The index from which to preview a step. This index is used to preview the result of steps that have already been applied, so that the resulting view frame is from earlier in the view frame stack.
- ViewFrame
-
- Type: ViewFrame structure
Represents the data being transformed during an action.
Result Syntax
[
'ActionId' => <integer>,
'Name' => '<string>',
'Result' => '<string>',
]
Result Details
Members
Errors
-
Updating or deleting a resource can cause an inconsistent state.
-
One or more resources can't be found.
-
The input parameters for this request failed validation.
StartJobRun
$result = $client->startJobRun([/* ... */]); $promise = $client->startJobRunAsync([/* ... */]);
Runs a DataBrew job.
Parameter Syntax
$result = $client->startJobRun([
'Name' => '<string>', // REQUIRED
]);
Parameter Details
Result Syntax
[
'RunId' => '<string>',
]
Result Details
Errors
-
Updating or deleting a resource can cause an inconsistent state.
-
One or more resources can't be found.
-
ServiceQuotaExceededException:
A service quota is exceeded.
-
The input parameters for this request failed validation.
StartProjectSession
$result = $client->startProjectSession([/* ... */]); $promise = $client->startProjectSessionAsync([/* ... */]);
Creates an interactive session, enabling you to manipulate data in a DataBrew project.
Parameter Syntax
$result = $client->startProjectSession([
'AssumeControl' => true || false,
'Name' => '<string>', // REQUIRED
]);
Parameter Details
Members
Result Syntax
[
'ClientSessionId' => '<string>',
'Name' => '<string>',
]
Result Details
Members
Errors
-
Updating or deleting a resource can cause an inconsistent state.
-
One or more resources can't be found.
-
ServiceQuotaExceededException:
A service quota is exceeded.
-
The input parameters for this request failed validation.
StopJobRun
$result = $client->stopJobRun([/* ... */]); $promise = $client->stopJobRunAsync([/* ... */]);
Stops a particular run of a job.
Parameter Syntax
$result = $client->stopJobRun([
'Name' => '<string>', // REQUIRED
'RunId' => '<string>', // REQUIRED
]);
Parameter Details
Members
Result Syntax
[
'RunId' => '<string>',
]
Result Details
Errors
-
One or more resources can't be found.
-
The input parameters for this request failed validation.
TagResource
$result = $client->tagResource([/* ... */]); $promise = $client->tagResourceAsync([/* ... */]);
Adds metadata tags to a DataBrew resource, such as a dataset, project, recipe, job, or schedule.
Parameter Syntax
$result = $client->tagResource([
'ResourceArn' => '<string>', // REQUIRED
'Tags' => ['<string>', ...], // REQUIRED
]);
Parameter Details
Members
- ResourceArn
-
- Required: Yes
- Type: string
The DataBrew resource to which tags should be added. The value for this parameter is an Amazon Resource Name (ARN). For DataBrew, you can tag a dataset, a job, a project, or a recipe.
- Tags
-
- Required: Yes
- Type: Associative array of custom strings keys (TagKey) to strings
One or more tags to be assigned to the resource.
Result Syntax
[]
Result Details
Errors
-
An internal service failure occurred.
-
One or more resources can't be found.
-
The input parameters for this request failed validation.
UntagResource
$result = $client->untagResource([/* ... */]); $promise = $client->untagResourceAsync([/* ... */]);
Removes metadata tags from a DataBrew resource.
Parameter Syntax
$result = $client->untagResource([
'ResourceArn' => '<string>', // REQUIRED
'TagKeys' => ['<string>', ...], // REQUIRED
]);
Parameter Details
Members
Result Syntax
[]
Result Details
Errors
-
An internal service failure occurred.
-
One or more resources can't be found.
-
The input parameters for this request failed validation.
UpdateDataset
$result = $client->updateDataset([/* ... */]); $promise = $client->updateDatasetAsync([/* ... */]);
Modifies the definition of an existing DataBrew dataset.
Parameter Syntax
$result = $client->updateDataset([
'Format' => 'CSV|JSON|PARQUET|EXCEL',
'FormatOptions' => [
'Csv' => [
'Delimiter' => '<string>',
'HeaderRow' => true || false,
],
'Excel' => [
'HeaderRow' => true || false,
'SheetIndexes' => [<integer>, ...],
'SheetNames' => ['<string>', ...],
],
'Json' => [
'MultiLine' => true || false,
],
],
'Input' => [ // REQUIRED
'DataCatalogInputDefinition' => [
'CatalogId' => '<string>',
'DatabaseName' => '<string>', // REQUIRED
'TableName' => '<string>', // REQUIRED
'TempDirectory' => [
'Bucket' => '<string>', // REQUIRED
'Key' => '<string>',
],
],
'DatabaseInputDefinition' => [
'DatabaseTableName' => '<string>', // REQUIRED
'GlueConnectionName' => '<string>', // REQUIRED
'TempDirectory' => [
'Bucket' => '<string>', // REQUIRED
'Key' => '<string>',
],
],
'S3InputDefinition' => [
'Bucket' => '<string>', // REQUIRED
'Key' => '<string>',
],
],
'Name' => '<string>', // REQUIRED
'PathOptions' => [
'FilesLimit' => [
'MaxFiles' => <integer>, // REQUIRED
'Order' => 'DESCENDING|ASCENDING',
'OrderedBy' => 'LAST_MODIFIED_DATE',
],
'LastModifiedDateCondition' => [
'Expression' => '<string>', // REQUIRED
'ValuesMap' => ['<string>', ...], // REQUIRED
],
'Parameters' => [
'<PathParameterName>' => [
'CreateColumn' => true || false,
'DatetimeOptions' => [
'Format' => '<string>', // REQUIRED
'LocaleCode' => '<string>',
'TimezoneOffset' => '<string>',
],
'Filter' => [
'Expression' => '<string>', // REQUIRED
'ValuesMap' => ['<string>', ...], // REQUIRED
],
'Name' => '<string>', // REQUIRED
'Type' => 'Datetime|Number|String', // REQUIRED
],
// ...
],
],
]);
Parameter Details
Members
- Format
-
- Type: string
The file format of a dataset that is created from an S3 file or folder.
- FormatOptions
-
- Type: FormatOptions structure
Represents a set of options that define the structure of either comma-separated value (CSV), Excel, or JSON input.
- Input
-
- Required: Yes
- Type: Input structure
Represents information on how DataBrew can find data, in either the AWS Glue Data Catalog or Amazon S3.
- Name
-
- Required: Yes
- Type: string
The name of the dataset to be updated.
- PathOptions
-
- Type: PathOptions structure
A set of options that defines how DataBrew interprets an S3 path of the dataset.
Result Syntax
[
'Name' => '<string>',
]
Result Details
Errors
-
Access to the specified resource was denied.
-
One or more resources can't be found.
-
The input parameters for this request failed validation.
UpdateProfileJob
$result = $client->updateProfileJob([/* ... */]); $promise = $client->updateProfileJobAsync([/* ... */]);
Modifies the definition of an existing profile job.
Parameter Syntax
$result = $client->updateProfileJob([
'EncryptionKeyArn' => '<string>',
'EncryptionMode' => 'SSE-KMS|SSE-S3',
'JobSample' => [
'Mode' => 'FULL_DATASET|CUSTOM_ROWS',
'Size' => <integer>,
],
'LogSubscription' => 'ENABLE|DISABLE',
'MaxCapacity' => <integer>,
'MaxRetries' => <integer>,
'Name' => '<string>', // REQUIRED
'OutputLocation' => [ // REQUIRED
'Bucket' => '<string>', // REQUIRED
'Key' => '<string>',
],
'RoleArn' => '<string>', // REQUIRED
'Timeout' => <integer>,
]);
Parameter Details
Members
- EncryptionKeyArn
-
- Type: string
The Amazon Resource Name (ARN) of an encryption key that is used to protect the job.
- EncryptionMode
-
- Type: string
The encryption mode for the job, which can be one of the following:
-
SSE-KMS- Server-side encryption with keys managed by AWS KMS. -
SSE-S3- Server-side encryption with keys managed by Amazon S3.
- JobSample
-
- Type: JobSample structure
Sample configuration for Profile Jobs only. Determines the number of rows on which the Profile job will be executed. If a JobSample value is not provided for profile jobs, the default value will be used. The default value is CUSTOM_ROWS for the mode parameter and 20000 for the size parameter.
- LogSubscription
-
- Type: string
Enables or disables Amazon CloudWatch logging for the job. If logging is enabled, CloudWatch writes one log stream for each job run.
- MaxCapacity
-
- Type: int
The maximum number of compute nodes that DataBrew can use when the job processes data.
- MaxRetries
-
- Type: int
The maximum number of times to retry the job after a job run fails.
- Name
-
- Required: Yes
- Type: string
The name of the job to be updated.
- OutputLocation
-
- Required: Yes
- Type: S3Location structure
Represents an Amazon S3 location (bucket name and object key) where DataBrew can read input data, or write output from a job.
- RoleArn
-
- Required: Yes
- Type: string
The Amazon Resource Name (ARN) of the AWS Identity and Access Management (IAM) role to be assumed when DataBrew runs the job.
- Timeout
-
- Type: int
The job's timeout in minutes. A job that attempts to run longer than this timeout period ends with a status of
TIMEOUT.
Result Syntax
[
'Name' => '<string>',
]
Result Details
Errors
-
Access to the specified resource was denied.
-
One or more resources can't be found.
-
The input parameters for this request failed validation.
UpdateProject
$result = $client->updateProject([/* ... */]); $promise = $client->updateProjectAsync([/* ... */]);
Modifies the definition of an existing DataBrew project.
Parameter Syntax
$result = $client->updateProject([
'Name' => '<string>', // REQUIRED
'RoleArn' => '<string>', // REQUIRED
'Sample' => [
'Size' => <integer>,
'Type' => 'FIRST_N|LAST_N|RANDOM', // REQUIRED
],
]);
Parameter Details
Members
- Name
-
- Required: Yes
- Type: string
The name of the project to be updated.
- RoleArn
-
- Required: Yes
- Type: string
The Amazon Resource Name (ARN) of the IAM role to be assumed for this request.
- Sample
-
- Type: Sample structure
Represents the sample size and sampling type for DataBrew to use for interactive data analysis.
Result Syntax
[
'LastModifiedDate' => <DateTime>,
'Name' => '<string>',
]
Result Details
Members
Errors
-
One or more resources can't be found.
-
The input parameters for this request failed validation.
UpdateRecipe
$result = $client->updateRecipe([/* ... */]); $promise = $client->updateRecipeAsync([/* ... */]);
Modifies the definition of the LATEST_WORKING version of a DataBrew recipe.
Parameter Syntax
$result = $client->updateRecipe([
'Description' => '<string>',
'Name' => '<string>', // REQUIRED
'Steps' => [
[
'Action' => [ // REQUIRED
'Operation' => '<string>', // REQUIRED
'Parameters' => ['<string>', ...],
],
'ConditionExpressions' => [
[
'Condition' => '<string>', // REQUIRED
'TargetColumn' => '<string>', // REQUIRED
'Value' => '<string>',
],
// ...
],
],
// ...
],
]);
Parameter Details
Members
- Description
-
- Type: string
A description of the recipe.
- Name
-
- Required: Yes
- Type: string
The name of the recipe to be updated.
- Steps
-
- Type: Array of RecipeStep structures
One or more steps to be performed by the recipe. Each step consists of an action, and the conditions under which the action should succeed.
Result Syntax
[
'Name' => '<string>',
]
Result Details
Errors
-
The input parameters for this request failed validation.
-
One or more resources can't be found.
UpdateRecipeJob
$result = $client->updateRecipeJob([/* ... */]); $promise = $client->updateRecipeJobAsync([/* ... */]);
Modifies the definition of an existing DataBrew recipe job.
Parameter Syntax
$result = $client->updateRecipeJob([
'EncryptionKeyArn' => '<string>',
'EncryptionMode' => 'SSE-KMS|SSE-S3',
'LogSubscription' => 'ENABLE|DISABLE',
'MaxCapacity' => <integer>,
'MaxRetries' => <integer>,
'Name' => '<string>', // REQUIRED
'Outputs' => [ // REQUIRED
[
'CompressionFormat' => 'GZIP|LZ4|SNAPPY|BZIP2|DEFLATE|LZO|BROTLI|ZSTD|ZLIB',
'Format' => 'CSV|JSON|PARQUET|GLUEPARQUET|AVRO|ORC|XML',
'FormatOptions' => [
'Csv' => [
'Delimiter' => '<string>',
],
],
'Location' => [ // REQUIRED
'Bucket' => '<string>', // REQUIRED
'Key' => '<string>',
],
'Overwrite' => true || false,
'PartitionColumns' => ['<string>', ...],
],
// ...
],
'RoleArn' => '<string>', // REQUIRED
'Timeout' => <integer>,
]);
Parameter Details
Members
- EncryptionKeyArn
-
- Type: string
The Amazon Resource Name (ARN) of an encryption key that is used to protect the job.
- EncryptionMode
-
- Type: string
The encryption mode for the job, which can be one of the following:
-
SSE-KMS- Server-side encryption with keys managed by AWS KMS. -
SSE-S3- Server-side encryption with keys managed by Amazon S3.
- LogSubscription
-
- Type: string
Enables or disables Amazon CloudWatch logging for the job. If logging is enabled, CloudWatch writes one log stream for each job run.
- MaxCapacity
-
- Type: int
The maximum number of nodes that DataBrew can consume when the job processes data.
- MaxRetries
-
- Type: int
The maximum number of times to retry the job after a job run fails.
- Name
-
- Required: Yes
- Type: string
The name of the job to update.
- Outputs
-
- Required: Yes
- Type: Array of Output structures
One or more artifacts that represent the output from running the job.
- RoleArn
-
- Required: Yes
- Type: string
The Amazon Resource Name (ARN) of the AWS Identity and Access Management (IAM) role to be assumed when DataBrew runs the job.
- Timeout
-
- Type: int
The job's timeout in minutes. A job that attempts to run longer than this timeout period ends with a status of
TIMEOUT.
Result Syntax
[
'Name' => '<string>',
]
Result Details
Errors
-
Access to the specified resource was denied.
-
One or more resources can't be found.
-
The input parameters for this request failed validation.
UpdateSchedule
$result = $client->updateSchedule([/* ... */]); $promise = $client->updateScheduleAsync([/* ... */]);
Modifies the definition of an existing DataBrew schedule.
Parameter Syntax
$result = $client->updateSchedule([
'CronExpression' => '<string>', // REQUIRED
'JobNames' => ['<string>', ...],
'Name' => '<string>', // REQUIRED
]);
Parameter Details
Members
- CronExpression
-
- Required: Yes
- Type: string
The date or dates and time or times when the jobs are to be run. For more information, see Cron expressions in the AWS Glue DataBrew Developer Guide.
- JobNames
-
- Type: Array of strings
The name or names of one or more jobs to be run for this schedule.
- Name
-
- Required: Yes
- Type: string
The name of the schedule to update.
Result Syntax
[
'Name' => '<string>',
]
Result Details
Errors
-
One or more resources can't be found.
-
ServiceQuotaExceededException:
A service quota is exceeded.
-
The input parameters for this request failed validation.
Shapes
AccessDeniedException
ConditionExpression
Description
Represents an individual condition that evaluates to true or false.
Conditions are used with recipe actions. The action is only performed for column values where the condition evaluates to true.
If a recipe requires more than one condition, then the recipe must specify multiple ConditionExpression elements. Each condition is applied to the rows in a dataset first, before the recipe action is performed.
Members
- Condition
-
- Required: Yes
- Type: string
A specific condition to apply to a recipe action. For more information, see Recipe structure in the AWS Glue DataBrew Developer Guide.
- TargetColumn
-
- Required: Yes
- Type: string
A column to apply this condition to.
- Value
-
- Type: string
A value that the condition must evaluate to for the condition to succeed.
ConflictException
Description
Updating or deleting a resource can cause an inconsistent state.
Members
CsvOptions
Description
Represents a set of options that define how DataBrew will read a comma-separated value (CSV) file when creating a dataset from that file.
Members
CsvOutputOptions
Description
Represents a set of options that define how DataBrew will write a comma-separated value (CSV) file.
Members
DataCatalogInputDefinition
Description
Represents how metadata stored in the AWS Glue Data Catalog is defined in a DataBrew dataset.
Members
- CatalogId
-
- Type: string
The unique identifier of the AWS account that holds the Data Catalog that stores the data.
- DatabaseName
-
- Required: Yes
- Type: string
The name of a database in the Data Catalog.
- TableName
-
- Required: Yes
- Type: string
The name of a database table in the Data Catalog. This table corresponds to a DataBrew dataset.
- TempDirectory
-
- Type: S3Location structure
An Amazon location that AWS Glue Data Catalog can use as a temporary directory.
DatabaseInputDefinition
Description
Connection information for dataset input files stored in a database.
Members
- DatabaseTableName
-
- Required: Yes
- Type: string
The table within the target database.
- GlueConnectionName
-
- Required: Yes
- Type: string
The AWS Glue Connection that stores the connection information for the target database.
- TempDirectory
-
- Type: S3Location structure
Represents an Amazon S3 location (bucket name and object key) where DataBrew can read input data, or write output from a job.
Dataset
Description
Represents a dataset that can be processed by DataBrew.
Members
- AccountId
-
- Type: string
The ID of the AWS account that owns the dataset.
- CreateDate
-
- Type: timestamp (string|DateTime or anything parsable by strtotime)
The date and time that the dataset was created.
- CreatedBy
-
- Type: string
The Amazon Resource Name (ARN) of the user who created the dataset.
- Format
-
- Type: string
The file format of a dataset that is created from an S3 file or folder.
- FormatOptions
-
- Type: FormatOptions structure
A set of options that define how DataBrew interprets the data in the dataset.
- Input
-
- Required: Yes
- Type: Input structure
Information on how DataBrew can find the dataset, in either the AWS Glue Data Catalog or Amazon S3.
- LastModifiedBy
-
- Type: string
The Amazon Resource Name (ARN) of the user who last modified the dataset.
- LastModifiedDate
-
- Type: timestamp (string|DateTime or anything parsable by strtotime)
The last modification date and time of the dataset.
- Name
-
- Required: Yes
- Type: string
The unique name of the dataset.
- PathOptions
-
- Type: PathOptions structure
A set of options that defines how DataBrew interprets an S3 path of the dataset.
- ResourceArn
-
- Type: string
The unique Amazon Resource Name (ARN) for the dataset.
- Source
-
- Type: string
The location of the data for the dataset, either Amazon S3 or the AWS Glue Data Catalog.
- Tags
-
- Type: Associative array of custom strings keys (TagKey) to strings
Metadata tags that have been applied to the dataset.
DatasetParameter
Description
Represents a dataset paramater that defines type and conditions for a parameter in the S3 path of the dataset.
Members
- CreateColumn
-
- Type: boolean
Optional boolean value that defines whether the captured value of this parameter should be loaded as an additional column in the dataset.
- DatetimeOptions
-
- Type: DatetimeOptions structure
Additional parameter options such as a format and a timezone. Required for datetime parameters.
- Filter
-
- Type: FilterExpression structure
The optional filter expression structure to apply additional matching criteria to the parameter.
- Name
-
- Required: Yes
- Type: string
The name of the parameter that is used in the dataset's S3 path.
- Type
-
- Required: Yes
- Type: string
The type of the dataset parameter, can be one of a 'String', 'Number' or 'Datetime'.
DatetimeOptions
Description
Represents additional options for correct interpretation of datetime parameters used in the S3 path of a dataset.
Members
- Format
-
- Required: Yes
- Type: string
Required option, that defines the datetime format used for a date parameter in the S3 path. Should use only supported datetime specifiers and separation characters, all litera a-z or A-Z character should be escaped with single quotes. E.g. "MM.dd.yyyy-'at'-HH:mm".
- LocaleCode
-
- Type: string
Optional value for a non-US locale code, needed for correct interpretation of some date formats.
- TimezoneOffset
-
- Type: string
Optional value for a timezone offset of the datetime parameter value in the S3 path. Shouldn't be used if Format for this parameter includes timezone fields. If no offset specified, UTC is assumed.
ExcelOptions
Description
Represents a set of options that define how DataBrew will interpret a Microsoft Excel file when creating a dataset from that file.
Members
- HeaderRow
-
- Type: boolean
A variable that specifies whether the first row in the file is parsed as the header. If this value is false, column names are auto-generated.
- SheetIndexes
-
- Type: Array of ints
One or more sheet numbers in the Excel file that will be included in the dataset.
- SheetNames
-
- Type: Array of strings
One or more named sheets in the Excel file that will be included in the dataset.
FilesLimit
Description
Represents a limit imposed on number of S3 files that should be selected for a dataset from a connected S3 path.
Members
- MaxFiles
-
- Required: Yes
- Type: int
The number of S3 files to select.
- Order
-
- Type: string
A criteria to use for S3 files sorting before their selection. By default uses DESCENDING order, i.e. most recent files are selected first. Anotherpossible value is ASCENDING.
- OrderedBy
-
- Type: string
A criteria to use for S3 files sorting before their selection. By default uses LAST_MODIFIED_DATE as a sorting criteria. Currently it's the only allowed value.
FilterExpression
Description
Represents a structure for defining parameter conditions.
Members
- Expression
-
- Required: Yes
- Type: string
The expression which includes condition names followed by substitution variables, possibly grouped and combined with other conditions. For example, "(starts_with :prefix1 or starts_with :prefix2) and (ends_with :suffix1 or ends_with :suffix2)". Substitution variables should start with ':' symbol.
- ValuesMap
-
- Required: Yes
- Type: Associative array of custom strings keys (ValueReference) to strings
The map of substitution variable names to their values used in this filter expression.
FormatOptions
Description
Represents a set of options that define the structure of either comma-separated value (CSV), Excel, or JSON input.
Members
- Csv
-
- Type: CsvOptions structure
Options that define how CSV input is to be interpreted by DataBrew.
- Excel
-
- Type: ExcelOptions structure
Options that define how Excel input is to be interpreted by DataBrew.
- Json
-
- Type: JsonOptions structure
Options that define how JSON input is to be interpreted by DataBrew.
Input
Description
Represents information on how DataBrew can find data, in either the AWS Glue Data Catalog or Amazon S3.
Members
- DataCatalogInputDefinition
-
- Type: DataCatalogInputDefinition structure
The AWS Glue Data Catalog parameters for the data.
- DatabaseInputDefinition
-
- Type: DatabaseInputDefinition structure
Connection information for dataset input files stored in a database.
- S3InputDefinition
-
- Type: S3Location structure
The Amazon S3 location where the data is stored.
InternalServerException
Job
Description
Represents all of the attributes of a DataBrew job.
Members
- AccountId
-
- Type: string
The ID of the AWS account that owns the job.
- CreateDate
-
- Type: timestamp (string|DateTime or anything parsable by strtotime)
The date and time that the job was created.
- CreatedBy
-
- Type: string
The Amazon Resource Name (ARN) of the user who created the job.
- DatasetName
-
- Type: string
A dataset that the job is to process.
- EncryptionKeyArn
-
- Type: string
The Amazon Resource Name (ARN) of an encryption key that is used to protect the job output. For more information, see Encrypting data written by DataBrew jobs
- EncryptionMode
-
- Type: string
The encryption mode for the job, which can be one of the following:
-
SSE-KMS- Server-side encryption with keys managed by AWS KMS. -
SSE-S3- Server-side encryption with keys managed by Amazon S3.
- JobSample
-
- Type: JobSample structure
A sample configuration for profile jobs only, which determines the number of rows on which the profile job is run. If a
JobSamplevalue isn't provided, the default value is used. The default value is CUSTOM_ROWS for the mode parameter and 20,000 for the size parameter. - LastModifiedBy
-
- Type: string
The Amazon Resource Name (ARN) of the user who last modified the job.
- LastModifiedDate
-
- Type: timestamp (string|DateTime or anything parsable by strtotime)
The modification date and time of the job.
- LogSubscription
-
- Type: string
The current status of Amazon CloudWatch logging for the job.
- MaxCapacity
-
- Type: int
The maximum number of nodes that can be consumed when the job processes data.
- MaxRetries
-
- Type: int
The maximum number of times to retry the job after a job run fails.
- Name
-
- Required: Yes
- Type: string
The unique name of the job.
- Outputs
-
- Type: Array of Output structures
One or more artifacts that represent output from running the job.
- ProjectName
-
- Type: string
The name of the project that the job is associated with.
- RecipeReference
-
- Type: RecipeReference structure
A set of steps that the job runs.
- ResourceArn
-
- Type: string
The unique Amazon Resource Name (ARN) for the job.
- RoleArn
-
- Type: string
The Amazon Resource Name (ARN) of the role to be assumed for this job.
- Tags
-
- Type: Associative array of custom strings keys (TagKey) to strings
Metadata tags that have been applied to the job.
- Timeout
-
- Type: int
The job's timeout in minutes. A job that attempts to run longer than this timeout period ends with a status of
TIMEOUT. - Type
-
- Type: string
The job type of the job, which must be one of the following:
-
PROFILE- A job to analyze a dataset, to determine its size, data types, data distribution, and more. -
RECIPE- A job to apply one or more transformations to a dataset.
JobRun
Description
Represents one run of a DataBrew job.
Members
- Attempt
-
- Type: int
The number of times that DataBrew has attempted to run the job.
- CompletedOn
-
- Type: timestamp (string|DateTime or anything parsable by strtotime)
The date and time when the job completed processing.
- DatasetName
-
- Type: string
The name of the dataset for the job to process.
- ErrorMessage
-
- Type: string
A message indicating an error (if any) that was encountered when the job ran.
- ExecutionTime
-
- Type: int
The amount of time, in seconds, during which a job run consumed resources.
- JobName
-
- Type: string
The name of the job being processed during this run.
- JobSample
-
- Type: JobSample structure
A sample configuration for profile jobs only, which determines the number of rows on which the profile job is run. If a
JobSamplevalue isn't provided, the default is used. The default value is CUSTOM_ROWS for the mode parameter and 20,000 for the size parameter. - LogGroupName
-
- Type: string
The name of an Amazon CloudWatch log group, where the job writes diagnostic messages when it runs.
- LogSubscription
-
- Type: string
The current status of Amazon CloudWatch logging for the job run.
- Outputs
-
- Type: Array of Output structures
One or more output artifacts from a job run.
- RecipeReference
-
- Type: RecipeReference structure
The set of steps processed by the job.
- RunId
-
- Type: string
The unique identifier of the job run.
- StartedBy
-
- Type: string
The Amazon Resource Name (ARN) of the user who initiated the job run.
- StartedOn
-
- Type: timestamp (string|DateTime or anything parsable by strtotime)
The date and time when the job run began.
- State
-
- Type: string
The current state of the job run entity itself.
JobSample
Description
A sample configuration for profile jobs only, which determines the number of rows on which the profile job is run. If a JobSample value isn't provided, the default is used. The default value is CUSTOM_ROWS for the mode parameter and 20,000 for the size parameter.
Members
- Mode
-
- Type: string
A value that determines whether the profile job is run on the entire dataset or a specified number of rows. This value must be one of the following:
-
FULL_DATASET - The profile job is run on the entire dataset.
-
CUSTOM_ROWS - The profile job is run on the number of rows specified in the
Sizeparameter.
- Size
-
- Type: long (int|float)
The
Sizeparameter is only required when the mode is CUSTOM_ROWS. The profile job is run on the specified number of rows. The maximum value for size is Long.MAX_VALUE.Long.MAX_VALUE = 9223372036854775807
JsonOptions
Description
Represents the JSON-specific options that define how input is to be interpreted by AWS Glue DataBrew.
Members
Output
Description
Represents options that specify how and where DataBrew writes the output generated by recipe jobs or profile jobs.
Members
- CompressionFormat
-
- Type: string
The compression algorithm used to compress the output text of the job.
- Format
-
- Type: string
The data format of the output of the job.
- FormatOptions
-
- Type: OutputFormatOptions structure
Represents options that define how DataBrew formats job output files.
- Location
-
- Required: Yes
- Type: S3Location structure
The location in Amazon S3 where the job writes its output.
- Overwrite
-
- Type: boolean
A value that, if true, means that any data in the location specified for output is overwritten with new output.
- PartitionColumns
-
- Type: Array of strings
The names of one or more partition columns for the output of the job.
OutputFormatOptions
Description
Represents a set of options that define the structure of comma-separated (CSV) job output.
Members
- Csv
-
- Type: CsvOutputOptions structure
Represents a set of options that define the structure of comma-separated value (CSV) job output.
PathOptions
Description
Represents a set of options that define how DataBrew selects files for a given S3 path in a dataset.
Members
- FilesLimit
-
- Type: FilesLimit structure
If provided, this structure imposes a limit on a number of files that should be selected.
- LastModifiedDateCondition
-
- Type: FilterExpression structure
If provided, this structure defines a date range for matching S3 objects based on their LastModifiedDate attribute in S3.
- Parameters
-
- Type: Associative array of custom strings keys (PathParameterName) to DatasetParameter structures
A structure that maps names of parameters used in the S3 path of a dataset to their definitions.
Project
Description
Represents all of the attributes of a DataBrew project.
Members
- AccountId
-
- Type: string
The ID of the AWS account that owns the project.
- CreateDate
-
- Type: timestamp (string|DateTime or anything parsable by strtotime)
The date and time that the project was created.
- CreatedBy
-
- Type: string
The Amazon Resource Name (ARN) of the user who crated the project.
- DatasetName
-
- Type: string
The dataset that the project is to act upon.
- LastModifiedBy
-
- Type: string
The Amazon Resource Name (ARN) of the user who last modified the project.
- LastModifiedDate
-
- Type: timestamp (string|DateTime or anything parsable by strtotime)
The last modification date and time for the project.
- Name
-
- Required: Yes
- Type: string
The unique name of a project.
- OpenDate
-
- Type: timestamp (string|DateTime or anything parsable by strtotime)
The date and time when the project was opened.
- OpenedBy
-
- Type: string
The Amazon Resource Name (ARN) of the user that opened the project for use.
- RecipeName
-
- Required: Yes
- Type: string
The name of a recipe that will be developed during a project session.
- ResourceArn
-
- Type: string
The Amazon Resource Name (ARN) for the project.
- RoleArn
-
- Type: string
The Amazon Resource Name (ARN) of the role that will be assumed for this project.
- Sample
-
- Type: Sample structure
The sample size and sampling type to apply to the data. If this parameter isn't specified, then the sample consists of the first 500 rows from the dataset.
- Tags
-
- Type: Associative array of custom strings keys (TagKey) to strings
Metadata tags that have been applied to the project.
Recipe
Description
Represents one or more actions to be performed on a DataBrew dataset.
Members
- CreateDate
-
- Type: timestamp (string|DateTime or anything parsable by strtotime)
The date and time that the recipe was created.
- CreatedBy
-
- Type: string
The Amazon Resource Name (ARN) of the user who created the recipe.
- Description
-
- Type: string
The description of the recipe.
- LastModifiedBy
-
- Type: string
The Amazon Resource Name (ARN) of the user who last modified the recipe.
- LastModifiedDate
-
- Type: timestamp (string|DateTime or anything parsable by strtotime)
The last modification date and time of the recipe.
- Name
-
- Required: Yes
- Type: string
The unique name for the recipe.
- ProjectName
-
- Type: string
The name of the project that the recipe is associated with.
- PublishedBy
-
- Type: string
The Amazon Resource Name (ARN) of the user who published the recipe.
- PublishedDate
-
- Type: timestamp (string|DateTime or anything parsable by strtotime)
The date and time when the recipe was published.
- RecipeVersion
-
- Type: string
The identifier for the version for the recipe. Must be one of the following:
-
Numeric version (
X.Y) -XandYstand for major and minor version numbers. The maximum length of each is 6 digits, and neither can be negative values. BothXandYare required, and "0.0" isn't a valid version. -
LATEST_WORKING- the most recent valid version being developed in a DataBrew project. -
LATEST_PUBLISHED- the most recent published version.
- ResourceArn
-
- Type: string
The Amazon Resource Name (ARN) for the recipe.
- Steps
-
- Type: Array of RecipeStep structures
A list of steps that are defined by the recipe.
- Tags
-
- Type: Associative array of custom strings keys (TagKey) to strings
Metadata tags that have been applied to the recipe.
RecipeAction
Description
Represents a transformation and associated parameters that are used to apply a change to a DataBrew dataset. For more information, see Recipe structure and Recipe actions reference.
Members
RecipeReference
Description
Represents the name and version of a DataBrew recipe.
Members
RecipeStep
Description
Represents a single step from a DataBrew recipe to be performed.
Members
- Action
-
- Required: Yes
- Type: RecipeAction structure
The particular action to be performed in the recipe step.
- ConditionExpressions
-
- Type: Array of ConditionExpression structures
One or more conditions that must be met for the recipe step to succeed.
All of the conditions in the array must be met. In other words, all of the conditions must be combined using a logical AND operation.
RecipeVersionErrorDetail
Description
Represents any errors encountered when attempting to delete multiple recipe versions.
Members
ResourceNotFoundException
S3Location
Description
Represents an Amazon S3 location (bucket name and object key) where DataBrew can read input data, or write output from a job.
Members
Sample
Description
Represents the sample size and sampling type for DataBrew to use for interactive data analysis.
Members
Schedule
Description
Represents one or more dates and times when a job is to run.
Members
- AccountId
-
- Type: string
The ID of the AWS account that owns the schedule.
- CreateDate
-
- Type: timestamp (string|DateTime or anything parsable by strtotime)
The date and time that the schedule was created.
- CreatedBy
-
- Type: string
The Amazon Resource Name (ARN) of the user who created the schedule.
- CronExpression
-
- Type: string
The dates and times when the job is to run. For more information, see Cron expressions in the AWS Glue DataBrew Developer Guide.
- JobNames
-
- Type: Array of strings
A list of jobs to be run, according to the schedule.
- LastModifiedBy
-
- Type: string
The Amazon Resource Name (ARN) of the user who last modified the schedule.
- LastModifiedDate
-
- Type: timestamp (string|DateTime or anything parsable by strtotime)
The date and time when the schedule was last modified.
- Name
-
- Required: Yes
- Type: string
The name of the schedule.
- ResourceArn
-
- Type: string
The Amazon Resource Name (ARN) of the schedule.
- Tags
-
- Type: Associative array of custom strings keys (TagKey) to strings
Metadata tags that have been applied to the schedule.
ServiceQuotaExceededException
ValidationException
ViewFrame
Description
Represents the data being transformed during an action.
Members
- ColumnRange
-
- Type: int
The number of columns to include in the view frame, beginning with the
StartColumnIndexvalue and ignoring any columns in theHiddenColumnslist. - HiddenColumns
-
- Type: Array of strings
A list of columns to hide in the view frame.
- StartColumnIndex
-
- Required: Yes
- Type: int
The starting index for the range of columns to return in the view frame.