Class: Aws::Glue::Types::DirectKinesisSource

Inherits:
Struct
  • Object
show all
Defined in:
gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb

Overview

Note:

When making an API call, you may pass DirectKinesisSource data as a hash:

{
  name: "NodeName", # required
  window_size: 1,
  detect_schema: false,
  streaming_options: {
    endpoint_url: "EnclosedInStringProperty",
    stream_name: "EnclosedInStringProperty",
    classification: "EnclosedInStringProperty",
    delimiter: "EnclosedInStringProperty",
    starting_position: "latest", # accepts latest, trim_horizon, earliest
    max_fetch_time_in_ms: 1,
    max_fetch_records_per_shard: 1,
    max_record_per_read: 1,
    add_idle_time_between_reads: false,
    idle_time_between_reads_in_ms: 1,
    describe_shard_interval: 1,
    num_retries: 1,
    retry_interval_ms: 1,
    max_retry_interval_ms: 1,
    avoid_empty_batches: false,
    stream_arn: "EnclosedInStringProperty",
    role_arn: "EnclosedInStringProperty",
    role_session_name: "EnclosedInStringProperty",
  },
  data_preview_options: {
    polling_time: 1,
    record_polling_limit: 1,
  },
}

Specifies a direct Amazon Kinesis data source.

Constant Summary collapse

SENSITIVE =
[]

Instance Attribute Summary collapse

Instance Attribute Details

#data_preview_optionsTypes::StreamingDataPreviewOptions

Additional options for data preview.



9528
9529
9530
9531
9532
9533
9534
9535
9536
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 9528

class DirectKinesisSource < Struct.new(
  :name,
  :window_size,
  :detect_schema,
  :streaming_options,
  :data_preview_options)
  SENSITIVE = []
  include Aws::Structure
end

#detect_schemaBoolean

Whether to automatically determine the schema from the incoming data.

Returns:

  • (Boolean)


9528
9529
9530
9531
9532
9533
9534
9535
9536
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 9528

class DirectKinesisSource < Struct.new(
  :name,
  :window_size,
  :detect_schema,
  :streaming_options,
  :data_preview_options)
  SENSITIVE = []
  include Aws::Structure
end

#nameString

The name of the data source.

Returns:

  • (String)


9528
9529
9530
9531
9532
9533
9534
9535
9536
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 9528

class DirectKinesisSource < Struct.new(
  :name,
  :window_size,
  :detect_schema,
  :streaming_options,
  :data_preview_options)
  SENSITIVE = []
  include Aws::Structure
end

#streaming_optionsTypes::KinesisStreamingSourceOptions

Additional options for the Kinesis streaming data source.



9528
9529
9530
9531
9532
9533
9534
9535
9536
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 9528

class DirectKinesisSource < Struct.new(
  :name,
  :window_size,
  :detect_schema,
  :streaming_options,
  :data_preview_options)
  SENSITIVE = []
  include Aws::Structure
end

#window_sizeInteger

The amount of time to spend processing each micro batch.

Returns:

  • (Integer)


9528
9529
9530
9531
9532
9533
9534
9535
9536
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 9528

class DirectKinesisSource < Struct.new(
  :name,
  :window_size,
  :detect_schema,
  :streaming_options,
  :data_preview_options)
  SENSITIVE = []
  include Aws::Structure
end