DatasetDefinitionProperty Struct
Configuration for Dataset Definition inputs.
The Dataset Definition input must specify exactly one of either AthenaDatasetDefinition or RedshiftDatasetDefinition types.
Example
ts
// The code below shows an example of how to instantiate this type.
// The values are placeholders you should change.
import { aws_sagemaker as sagemaker } from 'aws-cdk-lib';
const datasetDefinitionProperty: sagemaker.CfnProcessingJob.DatasetDefinitionProperty = {
athenaDatasetDefinition: {
catalog: 'catalog',
database: 'database',
outputFormat: 'outputFormat',
outputS3Uri: 'outputS3Uri',
queryString: 'queryString',
// the properties below are optional
kmsKeyId: 'kmsKeyId',
outputCompression: 'outputCompression',
workGroup: 'workGroup',
},
dataDistributionType: 'dataDistributionType',
inputMode: 'inputMode',
localPath: 'localPath',
redshiftDatasetDefinition: {
clusterId: 'clusterId',
clusterRoleArn: 'clusterRoleArn',
database: 'database',
dbUser: 'dbUser',
outputFormat: 'outputFormat',
outputS3Uri: 'outputS3Uri',
queryString: 'queryString',
// the properties below are optional
kmsKeyId: 'kmsKeyId',
outputCompression: 'outputCompression',
},
};
Properties
athenaDatasetDefinitionoptionalIResolvable | CfnProcessingJob.AthenaDatasetDefinitionPropertyathena_dataset_definitionoptionaltyping.Union[IResolvable, AthenaDatasetDefinitionProperty]athenaDatasetDefinitionoptionalIResolvableAthenaDatasetDefinitionoptionalIResolvableAthenaDatasetDefinitionoptionalinterface{}#Configuration for Athena Dataset Definition input.
dataDistributionTypeoptionalstringdata_distribution_typeoptionalstrdataDistributionTypeoptionalStringDataDistributionTypeoptionalstringDataDistributionTypeoptional*string#Whether the generated dataset is `FullyReplicated` or `ShardedByS3Key` (default).
inputModeoptionalstringinput_modeoptionalstrinputModeoptionalStringInputModeoptionalstringInputModeoptional*string#Whether to use `File` or `Pipe` input mode.
notes
In File (default) mode, Amazon SageMaker copies the data from the input source onto the local Amazon Elastic Block Store (Amazon EBS) volumes before starting your training algorithm. This is the most commonly used input mode. In Pipe mode, Amazon SageMaker streams input data from the source directly to your algorithm without using the EBS volume.
localPathoptionalstringlocal_pathoptionalstrlocalPathoptionalStringLocalPathoptionalstringLocalPathoptional*string#The local path where you want Amazon SageMaker to download the Dataset Definition inputs to run a processing job.
LocalPath is an absolute path to the input data. This is a required parameter when AppManaged is False (default).
redshiftDatasetDefinitionoptionalIResolvable | CfnProcessingJob.RedshiftDatasetDefinitionPropertyredshift_dataset_definitionoptionaltyping.Union[IResolvable, RedshiftDatasetDefinitionProperty]redshiftDatasetDefinitionoptionalIResolvableRedshiftDatasetDefinitionoptionalIResolvableRedshiftDatasetDefinitionoptionalinterface{}#Configuration for Redshift Dataset Definition input.