Auto-Record to Amazon S3 - Amazon Interactive Video Service

Auto-Record to Amazon S3

This section covers the storage structure created by the auto-record-to-S3 feature of Amazon Interactive Video Service (IVS). We explain the storage contents and metadata file schema.

S3 Prefix

The S3 prefix is a unique directory structure for each live stream that is recorded. All media and metadata files for the live stream are written within this prefix. For channels with recording enabled, the S3 prefix is generated when a live session starts and will be provided in the CloudWatch event at the start and end of a recording.

The S3 prefix has the following format:

/ivs/v1/<aws_account_id>/<channel_id>/<year>/<month>/<day>/<hours>/<minutes>/<recording_id>

Where:

  • aws_account_id is the ID of your AWS account (generated when you created an AWS account), from which the channel is created.

  • channel_id is the resource ID part of the channel ARN (the last part of the Amazon Resource Name). See ARN in the Glossary.

  • <year>/<month>/<day>/<hours>/<minutes> is a UTC timestamp when recording starts.

  • recording_id is a unique ID generated for each recording session.

For example:

ivs/v1/123456789012/AsXego4U6tnj/2020/6/23/20/12/j8Z9O91ndcVs

Recording Contents

When recording starts, video segments and metadata files are written to the S3 bucket that is configured for the channel. These contents are available for post-processing or playback as on-demand video.

Note that after a live stream starts and the Recording Start EventBridge event is emitted, it takes a little time before the manifest files and video segments are written. We recommend that you play back or process recorded streams only after the Recording End event is sent. (See Using Amazon EventBridge with Amazon IVS.)

The following is a sample directory structure and contents of a recording of a live Amazon IVS session:

ivs/v1/123456789012/AsXego4U6tnj/2020/6/23/20/12/j8Z9O91ndcVs/ events recording-started.json recording-ended.json media hls thumbnails

The events folder contains the metadata files corresponding to the recording event. JSON metadata files are generated when recording starts, ends successfully, or ends with failures:

  • events/recording-started.json

  • events/recording-ended.json

  • events/recording-failed.json

A given events folder will contain recording-started.json and either recording-ended.json or recording-failed.json.

These contain metadata related to the recorded session and its output formats. JSON details are given below.

The media folder contains all supported media contents, in two subfolders:

  • hls contains all media and manifest files generated during the live session and is playable with the Amazon IVS player.

  • thumbnails contains thumbnail images generated during the live session. Thumbnails are generated and written to the bucket every minute.

Important: The contents within the media folder are dynamically generated and determined by the characteristics of the first received video segments; the folder contents may not represent the ultimate characteristics (e.g., rendition quality). Do not make any assumptions about the static path. To discover the HLS renditions available and its path, use the JSON metadata files described in the following section.

JSON Metadata Files

When a recording state-change event occurs, a corresponding Amazon CloudWatch metric is generated and a metadata file is written within the S3 prefix. (See Monitoring Amazon IVS with Amazon CloudWatch.)

This metadata is in JSON format. It comprises the following information.

Field Type Required Description

channel_arn

string Yes ARN of the channel broadcasting the live stream.

media

object Yes

Object that contains the enumerated objects of media content available for this recording. Valid values: "hls", "thumbnails".

   

hls

object Yes Enumerated field that describes the Apple HLS format output.
      

duration_ms

integer Conditional

Duration of the recorded HLS content in milliseconds. This is available only when recording_status is "RECORDING_ENDED" or "RECORDING_ENDED_WITH_FAILURE". If a failure occurred before any recording was done, this is 0.

      

path

string Yes Relative path from the S3 prefix where HLS content is stored.
      

playlist

string Yes

Name of the HLS master playlist file.

      

renditions

object Yes Array of renditions (HLS variant) of metadata objects. There always is at least one rendition.
         

path

string Yes Relative path from the S3 prefix where HLS content is stored for this rendition.
         

playlist

string Yes Name of the media playlist file for this rendition.
         

resolution_height

int Conditional Pixel resolution height of the encoded video. This is available only when the rendition contains a video track.
         

resolution_width

int Conditional Pixel resolution width of the encoded video. This is available only when the rendition contains a video track.
   

thumbnails

object Yes Enumerated field that describes thumbnails output.
      

path

string Yes Relative path from the S3 prefix where thumbnail content is stored.

recording_ended_at

string Conditional

RFC 3339 UTC timestamp when the recording ended. This is available only when recording_status is "RECORDING_ENDED" or "RECORDING_ENDED_WITH_FAILURE".

recording_started_at

string Yes RFC 3339 UTC timestamp when the recording started.

recording_status

string Yes

Status of the recording. Valid values: "RECORDING_STARTED", "RECORDING_ENDED", "RECORDING_ENDED_WITH_FAILURE".

recording_status_message

string Conditional

Descriptive information on the status. This is available only when recording_status is "RECORDING_ENDED" or "RECORDING_ENDED_WITH_FAILURE".

version

string Yes The version of the metadata schema.

Example: recording_started.json

{ "version" : "v1", "channel_arn" : "arn:aws:ivs:us-west-2:123456789012:channel/AsXego4U6tnj", "recording_started_at" : "2020-06-12T12:53:26Z", "recording_status : "RECORDING_STARTED", "media" : { "hls" : { "path" : "media/hls", "playlist" : "master.m3u8", "renditions" : [ { "path" : "480p30", "playlist" : "playlist.m3u8", "resolution_height" : 480, "resolution_width" : 852 }, { "path" : "360p30", "playlist" : "playlist.m3u8", "resolution_height" : 360, "resolution_width" : 640 }, { "path" : "160p30", "playlist" : "playlist.m3u8", "resolution_height" : 160, "resolution_width" : 284 }, { "path" : "720p60", "playlist" : "playlist.m3u8", "resolution_height" : 720, "resolution_width" : 1280 } ] }, "thumbnails" : { "path" : "media/thumbnails" } } }

Example: recording_ended.json

{ "version" : "v1", "channel_arn" : "arn:aws:ivs:us-west-2:123456789012:channel/AsXego4U6tnj", "recording_ended_at" : "2020-06-14T12:53:20Z", "recording_started_at" : "2020-06-12T12:53:26Z", "recording_status" : "RECORDING_ENDED", "media" : { "hls" : { "duration_ms" : 172794489, "path" : "media/hls", "playlist" : "master.m3u8", "renditions" : [ { "path" : "480p30", "playlist" : "playlist.m3u8", "resolution_height" : 480, "resolution_width" : 852 }, { "path" : "360p30", "playlist" : "playlist.m3u8", "resolution_height" : 360, "resolution_width" : 640 }, { "path" : "160p30", "playlist" : "playlist.m3u8", "resolution_height" : 160, "resolution_width" : 284 }, { "path" : "720p60", "playlist" : "playlist.m3u8", "resolution_height" : 720, "resolution_width" : 1280 } ] }, "thumbnails" : { "path" : "media/thumbnails" } } }

Example: recording_failed.json

{ "version" : "v1", "channel_arn" : "arn:aws:ivs:us-west-2:123456789012:channel/AsXego4U6tnj", "recording_ended_at" : "2020-06-14T12:53:20Z", "recording_started_at" : "2020-06-12T12:53:26Z", "recording_status" : "RECORDING_ENDED_WITH_FAILURE", "recording_status_message": "InternalServerException", "media" : { "hls" : { "duration_ms" : 172794489, "path" : "media/hls", "playlist" : "master.m3u8", "renditions" : [ { "path" : "480p30", "playlist" : "playlist.m3u8", "resolution_height" : 480, "resolution_width" : 852 }, { "path" : "720p60", "playlist" : "playlist.m3u8", "resolution_height" : 720, "resolution_width" : 1280 } ] }, "thumbnails" : { "path" : "media/thumbnails" } } }

Discovering the Renditions of a Recording

When you stream content to an Amazon IVS channel, auto-record-to-s3 uses the source video to generate multiple renditions. Using Adaptive Bitrate Streaming (ABR), the Amazon IVS Player automatically switches the renditions (bitrates) as needed to optimize playback for varying network conditions.

Each rendition generated during live streaming is recorded in a unique path within the S3 recording prefix. The resolution detail, path, and playlist file names are stored in a JSON metadata file during the start and stop of the recording.

Important: Do not make any assumptions about the static rendition path or the list of generated renditions, as these are subject to change. Do not assume that a specific rendition will always be available for an Amazon IVS recording. To determine the available renditions, resolutions, and paths, refer to the metadata files.

The event/recording_started.json or event/recording_ended.json file within the recording prefix contains the paths and names of media files within the recording prefix. All path elements are relative to the previous path in the hierarchy. Elements under media > hls describe HLS assets, with master playlist name and path defined at this level.

Here is a Python code snippet that shows how to generate a master playlist path using the S3 recording prefix and metadata file:

def get_master_playlist(metadata_json, s3_recording_prefix): return s3_recording_prefix + '/' + metadata_json['media']['hls']['path'] + '/' + metadata_json['media']['hls']['playlist']

Elements under media > hls > renditions describe the list of renditions recorded. The resolution_height and resolution_width properties can be used to identify the video resolution. The path and playlist elements can be used to derive the rendition playlist path. Use these fields to determine which rendition to use for any post processing.

To discover the highest available rendition playlist for a recording, you can subscribe to "IVS Recording State Change" EventBridge events. (See Using Amazon EventBridge with Amazon IVS. Below is a sample Python script that illustrates using a lambda function subscribed to those events.

import json import boto3 s3 = boto3.resource('s3') def get_highest_rendition_playlist(bucket_name, prefix_name): object_path = "{}/events/recording-started.json".format(prefix_name) object = s3.Object(bucket_name, object_path) body = str(object.get()['Body'].read().decode('utf-8')) metadata = json.loads(body) media_path = metadata["media"]["hls"]["path"] renditions = metadata["media"]["hls"]["renditions"] highest_rendition = None highest_rendition_size = 0 for rendition in renditions: current_rendition_size = rendition["resolution_height"] if (current_rendition_size > highest_rendition_size): highest_rendition_size = current_rendition_size highest_rendition = rendition highest_rendition_playlist = media_path + '/' + highest_rendition['path'] + '/' + highest_rendition['playlist'] return highest_rendition_playlist def lambda_handler(event, context): prefix_name = event["detail"]["recording_s3_key_prefix"] bucket_name = event["detail"]["recording_s3_bucket_name"] rendition_playlist = get_highest_rendition_playlist(bucket_name, prefix_name) print("Highest rendition playlist: {}/{}".format(prefix_name, rendition_playlist)) return { 'statusCode': 200, 'body': rendition_playlist }