Jobs - AWS Data Exchange

Jobs

AWS Data Exchange jobs are asynchronous import or export operations used to create or copy assets. A data set owner can both import and export as they see fit. Someone with an entitlement to a data set can only export. Jobs are deleted 90 days after they are created.

This section defines the ListJobs and CreateJob operations.

URI

/v1/jobs

HTTP methods

GET

Operation ID: ListJobs

This operation lists your jobs sorted by CreatedAt in descending order.

Query parameters
Name Type Required Description
nextToken String False

The token value retrieved from a previous call to access the next page of results.

maxResults String False

The maximum number of results returned by a single call.

revisionId String False

The unique identifier for a revision.

dataSetId String False

The unique identifier for a data set.

Responses
Status code Response model Description
200 ListOfJobs

200 response

400 ValidationException

400 response

404 ResourceNotFoundException

404 response

429 ThrottlingException

429 response

500 InternalServerException

500 response

POST

Operation ID: CreateJob

This operation creates a job.

Responses
Status code Response model Description
201 Job

201 response

400 ValidationException

400 response

403 AccessDeniedException

403 response

404 ResourceNotFoundException

404 response

429 ThrottlingException

429 response

500 InternalServerException

500 response

Schemas

Request bodies

{ "Type": enum, "Details": { "ImportAssetsFromS3": { "AssetSources": [ { "Bucket": "string", "Key": "string" } ], "DataSetId": "string", "RevisionId": "string" }, "ExportAssetsToS3": { "DataSetId": "string", "RevisionId": "string", "AssetDestinations": [ { "Bucket": "string", "Key": "string", "AssetId": "string" } ], "Encryption": { "Type": enum, "KmsKeyArn": "string" } }, "ImportAssetFromSignedUrl": { "DataSetId": "string", "RevisionId": "string", "AssetName": "string", "Md5Hash": "string" }, "ExportAssetToSignedUrl": { "DataSetId": "string", "RevisionId": "string", "AssetId": "string" }, "ExportRevisionsToS3": { "DataSetId": "string", "RevisionDestinations": [ { "Bucket": "string", "KeyPattern": "string", "RevisionId": "string" } ], "Encryption": { "Type": enum, "KmsKeyArn": "string" } }, "ImportAssetsFromRedshiftDataShares": { "AssetSources": [ { "DataShareArn": "string" } ], "DataSetId": "string", "RevisionId": "string" } } }

Response bodies

{ "Jobs": [ { "Type": enum, "State": enum, "Errors": [ { "Code": enum, "Message": "string", "ResourceType": enum, "ResourceId": "string", "LimitName": enum, "LimitValue": number, "Details": { "ImportAssetsFromS3JobErrorDetails": [ { "Bucket": "string", "Key": "string" } ], "ImportAssetFromSignedUrlJobErrorDetails": { "AssetName": "string" } } } ], "Details": { "ImportAssetsFromS3": { "AssetSources": [ { "Bucket": "string", "Key": "string" } ], "DataSetId": "string", "RevisionId": "string" }, "ExportAssetsToS3": { "DataSetId": "string", "RevisionId": "string", "AssetDestinations": [ { "Bucket": "string", "Key": "string", "AssetId": "string" } ], "Encryption": { "Type": enum, "KmsKeyArn": "string" } }, "ImportAssetFromSignedUrl": { "DataSetId": "string", "RevisionId": "string", "AssetName": "string", "SignedUrl": "string", "SignedUrlExpiresAt": "string", "Md5Hash": "string" }, "ExportAssetToSignedUrl": { "DataSetId": "string", "RevisionId": "string", "AssetId": "string", "SignedUrl": "string", "SignedUrlExpiresAt": "string" }, "ExportRevisionsToS3": { "DataSetId": "string", "RevisionDestinations": [ { "Bucket": "string", "KeyPattern": "string", "RevisionId": "string" } ], "Encryption": { "Type": enum, "KmsKeyArn": "string" }, "EventActionArn": "string" }, "ImportAssetsFromRedshiftDataShares": { "AssetSources": [ { "DataShareArn": "string" } ], "DataSetId": "string", "RevisionId": "string" } }, "CreatedAt": "string", "UpdatedAt": "string", "Arn": "string", "Id": "string" } ], "NextToken": "string" }
{ "Type": enum, "State": enum, "Errors": [ { "Code": enum, "Message": "string", "ResourceType": enum, "ResourceId": "string", "LimitName": enum, "LimitValue": number, "Details": { "ImportAssetsFromS3JobErrorDetails": [ { "Bucket": "string", "Key": "string" } ], "ImportAssetFromSignedUrlJobErrorDetails": { "AssetName": "string" } } } ], "Details": { "ImportAssetsFromS3": { "AssetSources": [ { "Bucket": "string", "Key": "string" } ], "DataSetId": "string", "RevisionId": "string" }, "ExportAssetsToS3": { "DataSetId": "string", "RevisionId": "string", "AssetDestinations": [ { "Bucket": "string", "Key": "string", "AssetId": "string" } ], "Encryption": { "Type": enum, "KmsKeyArn": "string" } }, "ImportAssetFromSignedUrl": { "DataSetId": "string", "RevisionId": "string", "AssetName": "string", "SignedUrl": "string", "SignedUrlExpiresAt": "string", "Md5Hash": "string" }, "ExportAssetToSignedUrl": { "DataSetId": "string", "RevisionId": "string", "AssetId": "string", "SignedUrl": "string", "SignedUrlExpiresAt": "string" }, "ExportRevisionsToS3": { "DataSetId": "string", "RevisionDestinations": [ { "Bucket": "string", "KeyPattern": "string", "RevisionId": "string" } ], "Encryption": { "Type": enum, "KmsKeyArn": "string" }, "EventActionArn": "string" }, "ImportAssetsFromRedshiftDataShares": { "AssetSources": [ { "DataShareArn": "string" } ], "DataSetId": "string", "RevisionId": "string" } }, "CreatedAt": "string", "UpdatedAt": "string", "Arn": "string", "Id": "string" }
{ "Message": "string", "ExceptionCause": enum }
{ "Message": "string" }
{ "Message": "string", "ResourceType": enum, "ResourceId": "string" }
{ "Message": "string" }
{ "Message": "string" }

Properties

AccessDeniedException

Access to the resource is denied.

Property Type Required Description
Message

string

True

Access to the resource is denied.

AssetDestinationEntry

The destination for the asset.

Property Type Required Description
Bucket

string

True

The S3 bucket that is the destination for the asset.

Key

string

False

The name of the object in Amazon S3 for the asset.

AssetId

string

True

The unique identifier for the asset.

AssetSourceEntry

The source of the assets.

Property Type Required Description
Bucket

string

True

The S3 bucket that's part of the source of the asset.

Key

string

True

The name of the object in Amazon S3 for the asset.

CreateJobRequest

The CreateJob request. AWS Data Exchange Jobs are asynchronous import or export operations used to create or copy assets. A data set owner can both import and export assets. A subscriber with an entitlement to a data set can only export. Jobs are deleted 90 days after they are created. Created jobs must be started with the StartJob operation.

Property Type Required Description
Type

string

Values: IMPORT_ASSETS_FROM_S3 | IMPORT_ASSET_FROM_SIGNED_URL | EXPORT_ASSETS_TO_S3 | EXPORT_ASSET_TO_SIGNED_URL | EXPORT_REVISIONS_TO_S3 | IMPORT_ASSETS_FROM_REDSHIFT_DATA_SHARES

True

The type of job to be created.

Details

RequestDetails

True

The details for the CreateJob request.

Details

Information about the job error.

Property Type Required Description
ImportAssetsFromS3JobErrorDetails

Array of type AssetSourceEntry

False

Information about the job error.

ImportAssetFromSignedUrlJobErrorDetails

ImportAssetFromSignedUrlJobErrorDetails

False

Information about the job error.

ExportAssetToSignedUrlRequestDetails

Details of the operation to be performed by the job.

Property Type Required Description
DataSetId

string

True

The unique identifier for the data set associated with this export job.

RevisionId

string

True

The unique identifier for the revision associated with this export request.

AssetId

string

True

The unique identifier for the asset that is exported to a signed URL.

ExportAssetToSignedUrlResponseDetails

The details of the export to signed URL response.

Property Type Required Description
DataSetId

string

True

The unique identifier for the data set associated with this export job.

RevisionId

string

True

The unique identifier for the revision associated with this export response.

AssetId

string

True

The unique identifier for the asset associated with this export job.

SignedUrl

string

False

The signed URL for the export request.

SignedUrlExpiresAt

string

False

The date and time that the signed URL expires, in ISO 8601 format.

ExportAssetsToS3RequestDetails

Details of the operation to be performed by the job.

Property Type Required Description
DataSetId

string

True

The unique identifier for the data set associated with this export job.

RevisionId

string

True

The unique identifier for the revision associated with this export request.

AssetDestinations

Array of type AssetDestinationEntry

True

The destination for the asset.

Encryption

ExportServerSideEncryption

False

Encryption configuration for the export job.

ExportAssetsToS3ResponseDetails

Details about the export to Amazon S3 response.

Property Type Required Description
DataSetId

string

True

The unique identifier for the data set associated with this export job.

RevisionId

string

True

The unique identifier for the revision associated with this export response.

AssetDestinations

Array of type AssetDestinationEntry

True

The destination in Amazon S3 where the asset is exported.

Encryption

ExportServerSideEncryption

False

Encryption configuration of the export job.

ExportRevisionsToS3RequestDetails

Details of the operation to be performed by the job.

Property Type Required Description
DataSetId

string

True

The unique identifier for the data set associated with this export job.

RevisionDestinations

Array of type RevisionDestinationEntry

True

The destination for the revision.

Encryption

ExportServerSideEncryption

False

Encryption configuration for the export job.

ExportRevisionsToS3ResponseDetails

Details about the export revisions to Amazon S3 response.

Property Type Required Description
DataSetId

string

True

The unique identifier for the data set associated with this export job.

RevisionDestinations

Array of type RevisionDestinationEntry

True

The destination in Amazon S3 where the revision is exported.

Encryption

ExportServerSideEncryption

False

Encryption configuration of the export job.

EventActionArn

string

False

The Amazon Resource Name (ARN) of the event action.

ExportServerSideEncryption

Encryption configuration of the export job. Includes the encryption type in addition to the AWS KMS key. The KMS key is only necessary if you chose the KMS encryption. type.

Property Type Required Description
Type

ServerSideEncryptionTypes

True

The type of server side encryption used for encrypting the objects in Amazon S3.

KmsKeyArn

string

False

The Amazon Resource Name (ARN) of the AWS KMS key you want to use to encrypt the Amazon S3 objects. This parameter is required if you choose aws:kms as an encryption type.

ImportAssetFromSignedUrlJobErrorDetails

Information about the job error.

Property Type Required Description
AssetName

string

True

Information about the job error.

ImportAssetFromSignedUrlRequestDetails

Details of the operation to be performed by the job.

Property Type Required Description
DataSetId

string

True

The unique identifier for the data set associated with this import job.

RevisionId

string

True

The unique identifier for the revision associated with this import request.

AssetName

string

True

The name of the asset. When importing from Amazon S3, the S3 object key is used as the asset name.

Md5Hash

string

Pattern: ^(?:[A-Za-z0-9+/]{4})*(?:[A-Za-z0-9+/]{2}==|[A-Za-z0-9+/]{3}=)?$

MinLength: 24

MaxLength: 24

True

The Base64-encoded Md5 hash for the asset, used to ensure the integrity of the file at that location.

ImportAssetFromSignedUrlResponseDetails

The details in the response for an import request, including the signed URL and other information.

Property Type Required Description
DataSetId

string

True

The unique identifier for the data set associated with this import job.

RevisionId

string

True

The unique identifier for the revision associated with this import response.

AssetName

string

True

The name for the asset associated with this import job.

SignedUrl

string

False

The signed URL.

SignedUrlExpiresAt

string

False

The time and date at which the signed URL expires, in ISO 8601 format.

Md5Hash

string

Pattern: ^(?:[A-Za-z0-9+/]{4})*(?:[A-Za-z0-9+/]{2}==|[A-Za-z0-9+/]{3}=)?$

MinLength: 24

MaxLength: 24

False

The Base64-encoded Md5 hash for the asset, used to ensure the integrity of the file at that location.

ImportAssetsFromRedshiftDataSharesRequestDetails

Details from an import from Amazon Redshift datashare request.

Property Type Required Description
AssetSources

Array of type RedshiftDataShareAssetSourceEntry

True

A list of Amazon Redshift datashare assets.

DataSetId

string

True

The unique identifier for the data set associated with this import job.

RevisionId

string

True

The unique identifier for the revision associated with this import job.

ImportAssetsFromRedshiftDataSharesResponseDetails

Details from an import from Amazon Redshift datashare response.

Property Type Required Description
AssetSources

Array of type RedshiftDataShareAssetSourceEntry

True

A list of Amazon Redshift datashare asset sources.

DataSetId

string

True

The unique identifier for the data set associated with this import job.

RevisionId

string

True

The unique identifier for the revision associated with this import job.

ImportAssetsFromS3RequestDetails

Details of the operation to be performed by the job.

Property Type Required Description
AssetSources

Array of type AssetSourceEntry

True

Is a list of S3 bucket and object key pairs.

DataSetId

string

True

The unique identifier for the data set associated with this import job.

RevisionId

string

True

The unique identifier for the revision associated with this import request.

ImportAssetsFromS3ResponseDetails

Details from an import from Amazon S3 response.

Property Type Required Description
AssetSources

Array of type AssetSourceEntry

True

Is a list of Amazon S3 bucket and object key pairs.

DataSetId

string

True

The unique identifier for the data set associated with this import job.

RevisionId

string

True

The unique identifier for the revision associated with this import response.

InternalServerException

An exception occurred with the service.

Property Type Required Description
Message

string

True

The message identifying the service exception that occurred.

Job

AWS Data Exchange jobs are asynchronous import or export operations used to create or copy assets. Jobs are deleted 90 days after they are created.

Property Type Required Description
Type

string

Values: IMPORT_ASSETS_FROM_S3 | IMPORT_ASSET_FROM_SIGNED_URL | EXPORT_ASSETS_TO_S3 | EXPORT_ASSET_TO_SIGNED_URL | EXPORT_REVISIONS_TO_S3 | IMPORT_ASSETS_FROM_REDSHIFT_DATA_SHARES

True

The job type.

State

string

Values: WAITING | IN_PROGRESS | ERROR | COMPLETED | CANCELLED | TIMED_OUT

True

The state of the job.

Errors

Array of type JobError

False

The errors associated with jobs.

Details

ResponseDetails

True

Details about the job.

CreatedAt

string

True

The date and time that the job was created, in ISO 8601 format.

UpdatedAt

string

True

The date and time that the job was last updated, in ISO 8601 format.

Arn

string

True

The ARN for the job.

Id

string

True

The unique identifier for the job.

JobEntry

AWS Data Exchange Jobs are asynchronous import or export operations used to create or copy assets. A data set owner can both import and export as they see fit. Someone with an entitlement to a data set can only export. Jobs are deleted 90 days after they are created.

Property Type Required Description
Type

string

Values: IMPORT_ASSETS_FROM_S3 | IMPORT_ASSET_FROM_SIGNED_URL | EXPORT_ASSETS_TO_S3 | EXPORT_ASSET_TO_SIGNED_URL | EXPORT_REVISIONS_TO_S3 | IMPORT_ASSETS_FROM_REDSHIFT_DATA_SHARES

True

The job type.

State

string

Values: WAITING | IN_PROGRESS | ERROR | COMPLETED | CANCELLED | TIMED_OUT

True

The state of the job.

Errors

Array of type JobError

False

Errors for jobs.

Details

ResponseDetails

True

Details of the operation to be performed by the job, such as export destination details or import source details.

CreatedAt

string

True

The date and time that the job was created, in ISO 8601 format.

UpdatedAt

string

True

The date and time that the job was last updated, in ISO 8601 format.

Arn

string

True

The ARN for the job.

Id

string

True

The unique identifier for the job.

JobError

An error that occurred with the job request.

Property Type Required Description
Code

string

Values: ACCESS_DENIED_EXCEPTION | INTERNAL_SERVER_EXCEPTION | MALWARE_DETECTED | RESOURCE_NOT_FOUND_EXCEPTION | SERVICE_QUOTA_EXCEEDED_EXCEPTION | VALIDATION_EXCEPTION | MALWARE_SCAN_ENCRYPTED_FILE

True

The code for the job error.

Message

string

True

The message related to the job error.

ResourceType

JobErrorResourceTypes

False

The type of resource related to the error.

ResourceId

string

False

The unique identifier for the resource related to the error.

LimitName

JobErrorLimitName

False

The name of the limit that was reached.

LimitValue

number

Format: float

False

The value of the exceeded limit.

Details

Details

False

The details about the job error.

JobErrorLimitName

The name of the limit that was reached.

  • Assets per revision

  • Asset size in GB

  • Amazon Redshift datashare assets per revision

JobErrorResourceTypes

The types of resource which the job error can apply to.

  • REVISION

  • ASSET

  • DATA_SET

ListOfJobs

The token value retrieved from a previous call to access the next page of results.

Property Type Required Description
Jobs

Array of type JobEntry

True

The jobs listed by the request.

NextToken

string

False

The token value retrieved from a previous call to access the next page of results.

RedshiftDataShareAssetSourceEntry

The source of the Amazon Redshift datashare asset.

Property Type Required Description
DataShareArn

string

True

The Amazon Resource Name (ARN) of the datashare asset.

RequestDetails

The details for the request.

Property Type Required Description
ImportAssetsFromS3

ImportAssetsFromS3RequestDetails

False

Details about the import from Amazon S3 request.

ExportAssetsToS3

ExportAssetsToS3RequestDetails

False

Details about the export to Amazon S3 request.

ImportAssetFromSignedUrl

ImportAssetFromSignedUrlRequestDetails

False

Details about the import from signed URL request.

ExportAssetToSignedUrl

ExportAssetToSignedUrlRequestDetails

False

Details about the export to signed URL request.

ExportRevisionsToS3

ExportRevisionsToS3RequestDetails

False

Details about the export to Amazon S3 request.

ImportAssetsFromRedshiftDataShares

ImportAssetsFromRedshiftDataSharesRequestDetails

False

Details from an import from Amazon Redshift datashare request.

ResourceNotFoundException

The resource couldn't be found.

Property Type Required Description
Message

string

True

The resource couldn't be found.

ResourceType

string

Values: DATA_SET | REVISION | ASSET | JOB | EVENT_ACTION

False

The type of resource that couldn't be found.

ResourceId

string

False

The unique identifier for the resource that couldn't be found.

ResponseDetails

Details for the response.

Property Type Required Description
ImportAssetsFromS3

ImportAssetsFromS3ResponseDetails

False

Details for the import from Amazon S3 response.

ExportAssetsToS3

ExportAssetsToS3ResponseDetails

False

Details for the export to Amazon S3 response.

ImportAssetFromSignedUrl

ImportAssetFromSignedUrlResponseDetails

False

Details for the import from signed URL response.

ExportAssetToSignedUrl

ExportAssetToSignedUrlResponseDetails

False

Details for the export to signed URL response.

ExportRevisionsToS3

ExportRevisionsToS3ResponseDetails

False

Details for the export revisions to Amazon S3 response.

ImportAssetsFromRedshiftDataShares

ImportAssetsFromRedshiftDataSharesResponseDetails

False

Details from an import from Amazon Redshift datashare response.

RevisionDestinationEntry

The destination where the assets in the revision will be exported.

Property Type Required Description
Bucket

string

True

The S3 bucket that is the destination for the assets in the revision.

KeyPattern

string

False

A string representing the pattern for generated names of the individual assets in the revision. For more information about key patterns, see Key patterns when exporting revisions.

RevisionId

string

True

The unique identifier for the revision.

ServerSideEncryptionTypes

The types of encryption supported in export jobs to Amazon S3.

  • aws:kms

  • AES256

ThrottlingException

The limit on the number of requests per second was exceeded.

Property Type Required Description
Message

string

True

The limit on the number of requests per second was exceeded.

ValidationException

The request was invalid.

Property Type Required Description
Message

string

True

The message that informs you about what was invalid about the request.

ExceptionCause

string

Values: InsufficientS3BucketPolicy | S3AccessDenied

False

The message that informs you about what the exception was.

See also

For more information about using this API in one of the language-specific AWS SDKs and references, see the following:

ListJobs

CreateJob