Menu
Amazon CloudWatch Events
User Guide

Scenario 5: Relay Events to an Amazon Kinesis Stream

In this scenario, you relay AWS API call events in CloudWatch Events to an Amazon Kinesis stream.

Step 1: Create an Amazon Kinesis Stream

To create an Amazon Kinesis stream

  1. Create a stream named "test". At a command prompt, type:

    aws kinesis create-stream --stream-name test --shard-count 1
  2. Check the progress of the stream's creation. At a command prompt, type:

    aws kinesis describe-stream --stream-name test

    Wait until you see ACTIVE in the stream status, like the one in the following example:

    {
        "StreamDescription": {
            "StreamStatus": "ACTIVE",
            "StreamName": "test",
            "StreamARN": "arn:aws:kinesis:us-east-1:123456789012:stream/test",
            "Shards": [
                {
                    "ShardId": "shardId-000000000000",
                    "HashKeyRange": {
                        "EndingHashKey": "170141183460469231731687303715884105727",
                        "StartingHashKey": "0"
                    },
                    "SequenceNumberRange": {
                        "StartingSequenceNumber": "49546986683135544286507457935754639466300920667981217794"
                    }
                }
            ]
        }
    }

Step 2: Create an Amazon CloudWatch Events Rule

To create a CloudWatch Events rule

  1. Open the CloudWatch console at https://console.aws.amazon.com/cloudwatch/.

  2. In the navigation pane, choose Events.

  3. Choose Create rule, and then under Event selector, choose AWS API call.

  4. In the Service name list, choose EC2.

  5. Choose the Specific operation radio button, and then in the Specific operation field, choose StopInstances from the list.

  6. Under Targets, choose Add target. In the Select target type list, choose Kinesis stream.

  7. In the Stream list, select the test stream that you created in "Step 1: Create an Amazon Kinesis Stream."

  8. Choose Configure input, and then choose one of the following options:

    1. Matched event—Sends all of the data fields in the event to CloudWatch Logs.

    2. Part of the matched event—Sends only the specified data field of the event to CloudWatch Logs. You specify the part of the event using a string formatted $.first_parameter.second_parameter. For example, to send just detail part of the event, type $.detail.

    3. Constant—Sends a JSON-formatted text string that you specify to CloudWatch Logs. For example, to send a text string for the event, type {"Name":"MyInstance"}. The constant must be valid JSON.

  9. Choose Configure details. On the Configure rule details screen, in the Name field, type a name for the rule.

  10. In the Description field, enter a brief description for your rule, for example, Relay the event to an Amazon Kinesis stream when an EC2 StopInstances API call is made.

  11. If you're satisfied with the rule, choose Create rule.

Step 3: Test Your Amazon CloudWatch Events Rule by Stopping an Instance

You can test your rule by stopping an Amazon EC2 instance using the Amazon EC2 console. After waiting a few minutes for the instance to stop, check your AWS Lambda metrics in the CloudWatch console to verify that your function was invoked.

To test your CloudWatch Events rule by stopping an instance

  1. Open the Amazon EC2 console at https://console.aws.amazon.com/ec2/.

  2. Launch an Amazon EC2 instance. For more information about how to launch an instance, see Launch Your Instance in the Amazon EC2 User Guide for Linux Instances.

  3. Stop an Amazon EC2 instance. For more information, see Stop and Start Your Instance in the Amazon EC2 User Guide for Linux Instances.

  4. To view your CloudWatch Events metrics, open the CloudWatch console https://console.aws.amazon.com/cloudwatch/.

  5. In the navigation pane, under Metrics, choose Events, and then choose ec2-start-stop-invocations to view the number of invocations on the graph. You should see one data point on the graph from the instance you just stopped.

Step 4: Get the Record to Verify that the Event is Relayed

To get the record to verify that the event was relayed

  1. Get an iterator to start reading from your Amazon Kinesis stream, At a command prompt, type:

    aws kinesis get-shard-iterator --shard-id shardId-000000000000 --shard-iterator-type TRIM_HORIZON --stream-name test

    If the command is successful, you should see output similar to the following example:

    {
        "ShardIterator": "AAAAAAAAAAHSywljv0zEgPX4NyKdZ5wryMzP9yALs8NeKbUjp1IxtZs1Sp+KEd9I6AJ9ZG4lNR1EMi+9Md/nHvtLyxpfhEzYvkTZ4D9DQVz/mBYWRO6OTZRKnW9gd+efGN2aHFdkH1rJl4BL9Wyrk+ghYG22D2T1Da2EyNSH1+LAbK33gQweTJADBdyMwlo5r6PqcP2dzhg="
    }
  2. Type the following command (use the shard iterator you obtained as the output of the previous command):

    aws kinesis get-records --shard-iterator AAAAAAAAAAHSywljv0zEgPX4NyKdZ5wryMzP9yALs8NeKbUjp1IxtZs1Sp+KEd9I6AJ9ZG4lNR1EMi+9Md/nHvtLyxpfhEzYvkTZ4D9DQVz/mBYWRO6OTZRKnW9gd+efGN2aHFdkH1rJl4BL9Wyrk+ghYG22D2T1Da2EyNSH1+LAbK33gQweTJADBdyMwlo5r6PqcP2dzhg=

    If the get-records command is successful, it will request records from your stream for the shard that you specified when you obtained the shard iterator, as in the following example:

    {
      "Records":[ {
        "Data":"<Base64EncodedEvent>",
        "PartitionKey":"123”,
        "ApproximateArrivalTimestamp": 1.441215410867E9,
        "SequenceNumber":"49544985256907370027570885864065577703022652638596431874"
      } ],
      "MillisBehindLatest":24000,
      "NextShardIterator":"AAAAAAAAAAEDOW3ugseWPE4503kqN1yN1UaodY8unE0sYslMUmC6lX9hlig5+t4RtZM0/tALfiI4QGjunVgJvQsjxjh2aLyxaAaPr+LaoENQ7eVs4EdYXgKyThTZGPcca2fVXYJWL3yafv9dsDwsYVedI66dbMZFC8rPMWc797zxQkv4pSKvPOZvrUIudb8UkH3VMzx58Is="
    }

    Note

    The GetRecords API is a request, which means you may receive zero or more records, even if there are records in your stream. Any records returned may not represent all the records currently in your stream. If you don't receive the data you're looking for, keep calling get-records until you see an output with empty records.

    Records in Amazon Kinesis are Base64 encoded. However, the streams support in the AWS CLI does not provide Base64 decoding. If you use a Base64 decoder to manually decode the data, <Base64EncodedEvent> you will see that it is the event relayed to the stream in JSON form.