Delivering Amazon Data Firehose streams to Apache Iceberg Tables in Amazon S3 is in preview and is subject to change.
Testing Firehose stream with sample data
You can use the AWS Management Console to ingest simulated stock ticker data. The console runs a script in your browser to put sample records in your Firehose stream. This enables you to test the configuration of your Firehose stream without having to generate your own test data.
The following is an example from the simulated data:
{"TICKER_SYMBOL":"QXZ","SECTOR":"HEALTHCARE","CHANGE":-0.05,"PRICE":84.51}
Note that standard Amazon Data Firehose charges apply when your Firehose stream transmits the data, but there is no charge when the data is generated. To stop incurring these charges, you can stop the sample stream from the console at any time.
Prerequisites
Before you begin, create a Firehose stream. For more information, see Tutorial: Create a Firehose stream from console.
Test with Amazon S3
Use the following procedure to test your Firehose stream with Amazon Simple Storage Service (Amazon S3) as the destination.
To test a Firehose stream using Amazon S3
-
Open the Firehose console at https://console.aws.amazon.com/firehose/
. -
Choose an active Firehose stream. The Firehose stream must be in Active status before you can start sending data.
-
Under Test with demo data, choose Start sending demo data to generate sample stock ticker data.
-
Follow the onscreen instructions to verify that data is being delivered to your S3 bucket. Note that it might take a few minutes for new objects to appear in your bucket, based on the buffering configuration of your bucket.
-
When the test is complete, choose Stop sending demo data to stop incurring usage charges.
Test with Amazon Redshift
Use the following procedure to test your Firehose stream with Amazon Redshift as the destination.
To test a Firehose stream using Amazon Redshift
-
Your Firehose stream expects a table to be present in your Amazon Redshift cluster. Connect to Amazon Redshift through a SQL interface and run the following statement to create a table that accepts the sample data.
create table firehose_test_table ( TICKER_SYMBOL varchar(4), SECTOR varchar(16), CHANGE float, PRICE float );
-
Open the Firehose console at https://console.aws.amazon.com/firehose/
. -
Choose an active Firehose stream. The Firehose stream must be in Active status before you can start sending data.
-
Edit the destination details for your Firehose stream to point to the newly created
firehose_test_table
table. -
Under Test with demo data, choose Start sending demo data to generate sample stock ticker data.
-
Follow the onscreen instructions to verify that data is being delivered to your table. Note that it might take a few minutes for new rows to appear in your table, based on the buffering configuration.
-
When the test is complete, choose Stop sending demo data to stop incurring usage charges.
-
Edit the destination details for your Firehose stream to point to another table.
-
(Optional) Delete the
firehose_test_table
table.
Test with OpenSearch Service
Use the following procedure to test your Firehose stream using Amazon OpenSearch Service as the destination.
To test a Firehose stream using OpenSearch Service
-
Open the Firehose console at https://console.aws.amazon.com/firehose/
. -
Choose an active Firehose stream. The Firehose stream must be in Active status before you can start sending data.
-
Under Test with demo data, choose Start sending demo data to generate sample stock ticker data.
-
Follow the onscreen instructions to verify that data is being delivered to your OpenSearch Service domain. For more information, see Searching Documents in an OpenSearch Service Domain in the Amazon OpenSearch Service Developer Guide.
-
When the test is complete, choose Stop sending demo data to stop incurring usage charges.
Test with Splunk
Use the following procedure to test your Firehose stream using Splunk as the destination.
To test a Firehose stream using Splunk
-
Open the Firehose console at https://console.aws.amazon.com/firehose/
. -
Choose an active Firehose stream. The Firehose stream must be in Active status before you can start sending data.
-
Under Test with demo data, choose Start sending demo data to generate sample stock ticker data.
-
Check whether the data is being delivered to your Splunk index. Example search terms in Splunk are
sourcetype="aws:firehose:json"
andindex="
. For more information about how to search for events in Splunk, see Search Manualname-of-your-splunk-index
"in the Splunk documentation. If the test data doesn't appear in your Splunk index, check your Amazon S3 bucket for failed events. Also see Data Not Delivered to Splunk.
-
When you finish testing, choose Stop sending demo data to stop incurring usage charges.
Test with Apache Iceberg Tables
Use the following procedure to test your Firehose stream with Apache Iceberg Tables as the destination.
To test a Firehose stream using Apache Iceberg Tables
-
Open the Firehose console at https://console.aws.amazon.com/firehose/
. -
Choose an active Firehose stream. The Firehose stream must be in Active status before you can start sending data.
-
Under Test with demo data, choose Start sending demo data to generate sample stock ticker data.
-
Follow the instructions on screen to verify that data is being delivered to your Apache Iceberg Tables. Note that it might take a few minutes for new objects to appear in your bucket, based on its buffering configuration.
-
If the test data doesn't appear in your Apache Iceberg Tables, check your Amazon S3 bucket for failed events.
-
When you finish testing, choose Stop sending demo data to stop incurring usage charges.