For new projects, we recommend that you use the new Managed Service for Apache Flink Studio over Kinesis Data Analytics for SQL Applications. Managed Service for Apache Flink Studio combines ease of use with advanced analytical capabilities, enabling you to build sophisticated stream processing applications in minutes.
DiscoverInputSchema
Note
This documentation is for version 1 of the Amazon Kinesis Data Analytics API, which only supports SQL applications. Version 2 of the API supports SQL and Java applications. For more information about version 2, see Amazon Kinesis Data Analytics API V2 Documentation.
Infers a schema by evaluating sample records on the specified streaming source (Amazon Kinesis stream or Amazon Kinesis Firehose delivery stream) or S3 object. In the response, the operation returns the inferred schema and also the sample records that the operation used to infer the schema.
You can use the inferred schema when configuring a streaming source for your application. For conceptual information, see Configuring Application Input. Note that when you create an application using the Amazon Kinesis Analytics console, the console uses this operation to infer a schema and show it in the console user interface.
This operation requires permissions to perform the
kinesisanalytics:DiscoverInputSchema
action.
Request Syntax
{
"InputProcessingConfiguration": {
"InputLambdaProcessor": {
"ResourceARN": "string
",
"RoleARN": "string
"
}
},
"InputStartingPositionConfiguration": {
"InputStartingPosition": "string
"
},
"ResourceARN": "string
",
"RoleARN": "string
",
"S3Configuration": {
"BucketARN": "string
",
"FileKey": "string
",
"RoleARN": "string
"
}
}
Request Parameters
The request accepts the following data in JSON format.
- InputProcessingConfiguration
-
The InputProcessingConfiguration to use to preprocess the records before discovering the schema of the records.
Type: InputProcessingConfiguration object
Required: No
- InputStartingPositionConfiguration
-
Point at which you want Amazon Kinesis Analytics to start reading records from the specified streaming source discovery purposes.
Type: InputStartingPositionConfiguration object
Required: No
- ResourceARN
-
Amazon Resource Name (ARN) of the streaming source.
Type: String
Length Constraints: Minimum length of 1. Maximum length of 2048.
Pattern:
arn:.*
Required: No
- RoleARN
-
ARN of the IAM role that Amazon Kinesis Analytics can assume to access the stream on your behalf.
Type: String
Length Constraints: Minimum length of 1. Maximum length of 2048.
Pattern:
arn:.*
Required: No
- S3Configuration
-
Specify this parameter to discover a schema from data in an Amazon S3 object.
Type: S3Configuration object
Required: No
Response Syntax
{
"InputSchema": {
"RecordColumns": [
{
"Mapping": "string",
"Name": "string",
"SqlType": "string"
}
],
"RecordEncoding": "string",
"RecordFormat": {
"MappingParameters": {
"CSVMappingParameters": {
"RecordColumnDelimiter": "string",
"RecordRowDelimiter": "string"
},
"JSONMappingParameters": {
"RecordRowPath": "string"
}
},
"RecordFormatType": "string"
}
},
"ParsedInputRecords": [
[ "string" ]
],
"ProcessedInputRecords": [ "string" ],
"RawInputRecords": [ "string" ]
}
Response Elements
If the action is successful, the service sends back an HTTP 200 response.
The following data is returned in JSON format by the service.
- InputSchema
-
Schema inferred from the streaming source. It identifies the format of the data in the streaming source and how each data element maps to corresponding columns in the in-application stream that you can create.
Type: SourceSchema object
- ParsedInputRecords
-
An array of elements, where each element corresponds to a row in a stream record (a stream record can have more than one row).
Type: Array of arrays of strings
- ProcessedInputRecords
-
Stream data that was modified by the processor specified in the
InputProcessingConfiguration
parameter.Type: Array of strings
- RawInputRecords
-
Raw stream data that was sampled to infer the schema.
Type: Array of strings
Errors
- InvalidArgumentException
-
Specified input parameter value is invalid.
HTTP Status Code: 400
- ResourceProvisionedThroughputExceededException
-
Discovery failed to get a record from the streaming source because of the Amazon Kinesis Streams ProvisionedThroughputExceededException. For more information, see GetRecords in the Amazon Kinesis Streams API Reference.
HTTP Status Code: 400
- ServiceUnavailableException
-
The service is unavailable. Back off and retry the operation.
HTTP Status Code: 500
- UnableToDetectSchemaException
-
Data format is not valid. Amazon Kinesis Analytics is not able to detect schema for the given streaming source.
HTTP Status Code: 400
- UnsupportedOperationException
-
The request was rejected because a specified parameter is not supported or a specified resource is not valid for this operation.
HTTP Status Code: 400
See Also
For more information about using this API in one of the language-specific AWS SDKs, see the following: