AWS SDK for Go (PILOT)
API Reference

PREVIEW DOCUMENTATION - This is a preview of a new format for the AWS SDK for Go API Reference documentation. For the current AWS SDK for Go API Reference, see https://docs.aws.amazon.com/sdk-for-go/api/.

We welcome your feedback on this new version of the documentation. Send your comments to aws-sdkdocs-feedback@amazon.com.

OrcSerDe

import "github.com/aws/aws-sdk-go/service/firehose"

type OrcSerDe struct { BlockSizeBytes *int64 `min:"6.7108864e+07" type:"integer"` BloomFilterColumns []*string `type:"list"` BloomFilterFalsePositiveProbability *float64 `type:"double"` Compression *string `type:"string" enum:"OrcCompression"` DictionaryKeyThreshold *float64 `type:"double"` EnablePadding *bool `type:"boolean"` FormatVersion *string `type:"string" enum:"OrcFormatVersion"` PaddingTolerance *float64 `type:"double"` RowIndexStride *int64 `min:"1000" type:"integer"` StripeSizeBytes *int64 `min:"8.388608e+06" type:"integer"` }

A serializer to use for converting data to the ORC format before storing it in Amazon S3. For more information, see Apache ORC (https://orc.apache.org/docs/).

BlockSizeBytes

Type: *int64

The Hadoop Distributed File System (HDFS) block size. This is useful if you intend to copy the data from Amazon S3 to HDFS before querying. The default is 256 MiB and the minimum is 64 MiB. Kinesis Data Firehose uses this value for padding calculations.

BloomFilterColumns

Type: []*string

The column names for which you want Kinesis Data Firehose to create bloom filters. The default is null.

BloomFilterFalsePositiveProbability

Type: *float64

The Bloom filter false positive probability (FPP). The lower the FPP, the bigger the Bloom filter. The default value is 0.05, the minimum is 0, and the maximum is 1.

Compression

Type: *string

The compression code to use over data blocks. The default is SNAPPY.

DictionaryKeyThreshold

Type: *float64

Represents the fraction of the total number of non-null rows. To turn off dictionary encoding, set this fraction to a number that is less than the number of distinct keys in a dictionary. To always use dictionary encoding, set this threshold to 1.

EnablePadding

Type: *bool

Set this to true to indicate that you want stripes to be padded to the HDFS block boundaries. This is useful if you intend to copy the data from Amazon S3 to HDFS before querying. The default is false.

FormatVersion

Type: *string

The version of the file to write. The possible values are V0_11 and V0_12. The default is V0_12.

PaddingTolerance

Type: *float64

A number between 0 and 1 that defines the tolerance for block padding as a decimal fraction of stripe size. The default value is 0.05, which means 5 percent of stripe size.

For the default values of 64 MiB ORC stripes and 256 MiB HDFS blocks, the default block padding tolerance of 5 percent reserves a maximum of 3.2 MiB for padding within the 256 MiB block. In such a case, if the available size within the block is more than 3.2 MiB, a new, smaller stripe is inserted to fit within that space. This ensures that no stripe crosses block boundaries and causes remote reads within a node-local task.

Kinesis Data Firehose ignores this parameter when OrcSerDe$EnablePadding is false.

RowIndexStride

Type: *int64

The number of rows between index entries. The default is 10,000 and the minimum is 1,000.

StripeSizeBytes

Type: *int64

The number of bytes in each stripe. The default is 64 MiB and the minimum is 8 MiB.

Method

GoString

func (s OrcSerDe) GoString() string

GoString returns the string representation

SetBlockSizeBytes

func (s *OrcSerDe) SetBlockSizeBytes(v int64) *OrcSerDe

SetBlockSizeBytes sets the BlockSizeBytes field's value.

SetBloomFilterColumns

func (s *OrcSerDe) SetBloomFilterColumns(v []*string) *OrcSerDe

SetBloomFilterColumns sets the BloomFilterColumns field's value.

SetBloomFilterFalsePositiveProbability

func (s *OrcSerDe) SetBloomFilterFalsePositiveProbability(v float64) *OrcSerDe

SetBloomFilterFalsePositiveProbability sets the BloomFilterFalsePositiveProbability field's value.

SetCompression

func (s *OrcSerDe) SetCompression(v string) *OrcSerDe

SetCompression sets the Compression field's value.

SetDictionaryKeyThreshold

func (s *OrcSerDe) SetDictionaryKeyThreshold(v float64) *OrcSerDe

SetDictionaryKeyThreshold sets the DictionaryKeyThreshold field's value.

SetEnablePadding

func (s *OrcSerDe) SetEnablePadding(v bool) *OrcSerDe

SetEnablePadding sets the EnablePadding field's value.

SetFormatVersion

func (s *OrcSerDe) SetFormatVersion(v string) *OrcSerDe

SetFormatVersion sets the FormatVersion field's value.

SetPaddingTolerance

func (s *OrcSerDe) SetPaddingTolerance(v float64) *OrcSerDe

SetPaddingTolerance sets the PaddingTolerance field's value.

SetRowIndexStride

func (s *OrcSerDe) SetRowIndexStride(v int64) *OrcSerDe

SetRowIndexStride sets the RowIndexStride field's value.

SetStripeSizeBytes

func (s *OrcSerDe) SetStripeSizeBytes(v int64) *OrcSerDe

SetStripeSizeBytes sets the StripeSizeBytes field's value.

String

func (s OrcSerDe) String() string

String returns the string representation

Validate

func (s *OrcSerDe) Validate() error

Validate inspects the fields of the type to determine if they are valid.

On this page: