Class: Aws::SageMaker::Types::Channel

Inherits:
Struct
  • Object
show all
Defined in:
gems/aws-sdk-sagemaker/lib/aws-sdk-sagemaker/types.rb

Overview

Note:

When making an API call, you may pass Channel data as a hash:

{
  channel_name: "ChannelName", # required
  data_source: { # required
    s3_data_source: {
      s3_data_type: "ManifestFile", # required, accepts ManifestFile, S3Prefix, AugmentedManifestFile
      s3_uri: "S3Uri", # required
      s3_data_distribution_type: "FullyReplicated", # accepts FullyReplicated, ShardedByS3Key
      attribute_names: ["AttributeName"],
    },
    file_system_data_source: {
      file_system_id: "FileSystemId", # required
      file_system_access_mode: "rw", # required, accepts rw, ro
      file_system_type: "EFS", # required, accepts EFS, FSxLustre
      directory_path: "DirectoryPath", # required
    },
  },
  content_type: "ContentType",
  compression_type: "None", # accepts None, Gzip
  record_wrapper_type: "None", # accepts None, RecordIO
  input_mode: "Pipe", # accepts Pipe, File, FastFile
  shuffle_config: {
    seed: 1, # required
  },
}

A channel is a named input source that training algorithms can consume.

See Also:

Constant Summary collapse

SENSITIVE = 
[]

Instance Attribute Summary collapse

Instance Attribute Details

#channel_name ⇒ String

The name of the channel.

Returns:

  • (String) 


2916
2917
2918
2919
2920
2921
2922
2923
2924
2925
2926
 
# File 'gems/aws-sdk-sagemaker/lib/aws-sdk-sagemaker/types.rb', line 2916

class Channel < Struct.new(
  :channel_name,
  :data_source,
  :content_type,
  :compression_type,
  :record_wrapper_type,
  :input_mode,
  :shuffle_config)
  SENSITIVE = []
  include Aws::Structure
end

#compression_type ⇒ String

If training data is compressed, the compression type. The default value is None. CompressionType is used only in Pipe input mode. In File mode, leave this field unset or set it to None.

Returns:

  • (String) 


2916
2917
2918
2919
2920
2921
2922
2923
2924
2925
2926
 
# File 'gems/aws-sdk-sagemaker/lib/aws-sdk-sagemaker/types.rb', line 2916

class Channel < Struct.new(
  :channel_name,
  :data_source,
  :content_type,
  :compression_type,
  :record_wrapper_type,
  :input_mode,
  :shuffle_config)
  SENSITIVE = []
  include Aws::Structure
end

#content_type ⇒ String

The MIME type of the data.

Returns:

  • (String) 


2916
2917
2918
2919
2920
2921
2922
2923
2924
2925
2926
 
# File 'gems/aws-sdk-sagemaker/lib/aws-sdk-sagemaker/types.rb', line 2916

class Channel < Struct.new(
  :channel_name,
  :data_source,
  :content_type,
  :compression_type,
  :record_wrapper_type,
  :input_mode,
  :shuffle_config)
  SENSITIVE = []
  include Aws::Structure
end

#data_source ⇒ Types::DataSource

The location of the channel data.

Returns:



2916
2917
2918
2919
2920
2921
2922
2923
2924
2925
2926
 
# File 'gems/aws-sdk-sagemaker/lib/aws-sdk-sagemaker/types.rb', line 2916

class Channel < Struct.new(
  :channel_name,
  :data_source,
  :content_type,
  :compression_type,
  :record_wrapper_type,
  :input_mode,
  :shuffle_config)
  SENSITIVE = []
  include Aws::Structure
end

#input_mode ⇒ String

(Optional) The input mode to use for the data channel in a training job. If you don't set a value for InputMode, Amazon SageMaker uses the value set for TrainingInputMode. Use this parameter to override the TrainingInputMode setting in a AlgorithmSpecification request when you have a channel that needs a different input mode from the training job's general setting. To download the data from Amazon Simple Storage Service (Amazon S3) to the provisioned ML storage volume, and mount the directory to a Docker volume, use File input mode. To stream data directly from Amazon S3 to the container, choose Pipe input mode.

To use a model for incremental training, choose File input model.

Returns:

  • (String) 


2916
2917
2918
2919
2920
2921
2922
2923
2924
2925
2926
 
# File 'gems/aws-sdk-sagemaker/lib/aws-sdk-sagemaker/types.rb', line 2916

class Channel < Struct.new(
  :channel_name,
  :data_source,
  :content_type,
  :compression_type,
  :record_wrapper_type,
  :input_mode,
  :shuffle_config)
  SENSITIVE = []
  include Aws::Structure
end

#record_wrapper_type ⇒ String

Specify RecordIO as the value when input data is in raw format but the training algorithm requires the RecordIO format. In this case, Amazon SageMaker wraps each individual S3 object in a RecordIO record. If the input data is already in RecordIO format, you don't need to set this attribute. For more information, see Create a Dataset Using RecordIO.

In File mode, leave this field unset or set it to None.

Returns:

  • (String) 


2916
2917
2918
2919
2920
2921
2922
2923
2924
2925
2926
 
# File 'gems/aws-sdk-sagemaker/lib/aws-sdk-sagemaker/types.rb', line 2916

class Channel < Struct.new(
  :channel_name,
  :data_source,
  :content_type,
  :compression_type,
  :record_wrapper_type,
  :input_mode,
  :shuffle_config)
  SENSITIVE = []
  include Aws::Structure
end

#shuffle_config ⇒ Types::ShuffleConfig

A configuration for a shuffle option for input data in a channel. If you use S3Prefix for S3DataType, this shuffles the results of the S3 key prefix matches. If you use ManifestFile, the order of the S3 object references in the ManifestFile is shuffled. If you use AugmentedManifestFile, the order of the JSON lines in the AugmentedManifestFile is shuffled. The shuffling order is determined using the Seed value.

For Pipe input mode, shuffling is done at the start of every epoch. With large datasets this ensures that the order of the training data is different for each epoch, it helps reduce bias and possible overfitting. In a multi-node training job when ShuffleConfig is combined with S3DataDistributionType of ShardedByS3Key, the data is shuffled across nodes so that the content sent to a particular node on the first epoch might be sent to a different node on the second epoch.

Returns:



2916
2917
2918
2919
2920
2921
2922
2923
2924
2925
2926
 
# File 'gems/aws-sdk-sagemaker/lib/aws-sdk-sagemaker/types.rb', line 2916

class Channel < Struct.new(
  :channel_name,
  :data_source,
  :content_type,
  :compression_type,
  :record_wrapper_type,
  :input_mode,
  :shuffle_config)
  SENSITIVE = []
  include Aws::Structure
end