Amazon Kinesis Video Streams
Developer Guide

Kinesis Video Streams Data Model

The Producer Libraries and Stream Parser Library send and receive video data in a format that supports embedding information alongside video data. This format is based on the Matroska (MKV) specification.

The MKV format is an open specification for media data. All the libraries and code examples in the Amazon Kinesis Video Streams Developer Guide send or receive data in the MKV format.

The Kinesis Video Streams Producer Libraries use the StreamDefinition and Frame types to produce MKV stream headers, frame headers, and frame data.

For information about the full MKV specification, see Matroska Specifications.

The following sections describe the components of MKV-formatted data produced by the C++ Producer Library.

Stream Header Elements

The following MKV header elements are used by StreamDefinition (defined in StreamDefinition.h).

Element Description Typical Values
stream_name Corresponds to the name of the Kinesis video stream. my-stream
retention_period The duration that stream data is persisted by Kinesis Video Streams. Specify 0 for a stream that does not retain data. 24
tags A key-value collection of user data. This data is displayed in the AWS Management Console and can be read by client applications to filter or get information about a stream.
kms_key_id If present, the user-defined AWS KMS master key that is used to encrypt data on the stream. If it is absent, the data is encrypted by the Kinesis-supplied master key (aws/kinesis-video). 01234567-89ab-cdef-0123-456789ab
streaming_type Currently, the only valid streaming type is STREAMING_TYPE_REALTIME. STREAMING_TYPE_REALTIME
content_type The user-defined content type. For streaming video data to play in the console, the content type must be video/h264. video/h264
max_latency This value is not currently used and should be set to 0. 0
fragment_duration The estimate of how long your fragments should be, which is used for optimization. The actual fragment duration is determined by the streaming data. 2
timecode_scale

Indicates the scale used by frame time stamps. The default is 1 millisecond. Specifying 0 also assigns the default value of 1 millisecond. This value can be between 100 nanoseconds and 1 second.

For more information, see TimecodeScale in the Matroska documentation.

key_frame_fragmentation If true, the stream starts a new cluster when a keyframe is received. true
frame_timecodes If true, Kinesis Video Streams stamps the frames when they are received. If false, Kinesis Video Streams uses the decode time of the received frames. true
absolute_fragment_time If true, the cluster timecodes are interpreted as using absolute time (for example, from the producer's system clock). If false, the cluster timecodes are interpreted as being relative to the start time of the stream. true
fragment_acks If true, acknowledgements (ACKs) are sent when Kinesis Video Streams receives the data. The ACKs can be received using the KinesisVideoStreamFragmentAck or KinesisVideoStreamParseFragmentAck callbacks. true
restart_on_error Indicates whether the stream should resume transmission after a stream error is raised. true
nal_adaptation_flags Indicates whether NAL (Network Abstraction Layer) adaptation or codec private data is present in the content. Valid flags include NAL_ADAPTATION_ANNEXB_NALS and NAL_ADAPTATION_ANNEXB_CPD_NALS. NAL_ADAPTATION_ANNEXB_NALS
frame_rate An estimate of the content frame rate. This value is used for optimization; the actual frame rate is determined by the rate of incoming data. Specifying 0 assigns the default of 24. 24
avg_bandwith_bps An estimate of the content bandwidth. This value is used for optimization; the actual rate is determined by the bandwidth of incoming data. For example, for a 720 p resolution video stream running at 25 FPS, you can expect the average bandwidth to be 5 Mbps. 5
buffer_duration The duration that content is to be buffered on the producer. If there is low network latency, this value can be reduced; if network latency is high, increasing this value prevents frames from being dropped before they can be sent, due to allocation failing to put frames into the smaller buffer.
replay_duration The amount of time the video data stream is "rewound" in the case of connection loss. This value can be zero if lost frames due to connection loss are not a concern; the value can be increased if the consuming application can eliminate redundant frames. This value should be less than the buffer duration; otherwise the buffer duration is used.
connection_staleness The duration that a connection is maintained when no data is received.
codec_id The codec used by the content. For more information, see CodecID in the Matroska specification. V_MPEG2
track_name The user-defined name of the track. my_track
codecPrivateData Data provided by the encoder used to decode the frame data, such as the frame width and height in pixels, which is needed by many downstream consumers. In the C++ Producer Library, the gMkvTrackVideoBits array in MkvStatics.cpp includes pixel width and height for the frame.
codecPrivateDataSize The size of the data in the codecPrivateData parameter.

Frame Header Elements

The following MKV header elements are used by Frame (defined in the KinesisVideoPic package, in mkvgen/Include.h):

  • Frame Index: A monotonically increasing value.

  • Flags: The type of frame. Valid values include the following:

    • FRAME_FLAGS_NONE

    • FRAME_FLAG_KEY_FRAME: If key_frame_fragmentation is set on the stream, key frames start a new fragment.

    • FRAME_FLAG_DISCARDABLE_FRAME: Tells the decoder that it can discard this frame if decoding is slow.

    • FRAME_FLAG_INVISIBLE_FRAME: Duration of this block is 0.

  • Decoding Timestamp: The time stamp of when this frame was decoded. If previous frames depend on this frame for decoding, this time stamp might be earlier than that of earlier frames. This value is relative to the start of the fragment.

  • Presentation Timestamp: The time stamp of when this frame is displayed. This value is relative to the start of the fragment.

  • Duration: The playback duration of the frame.

  • Size: The size of the frame data in bytes

MKV Frame Data

The data in frame.frameData might contain only media data for the frame, or it might contain further nested header information, depending on the encoding schema used. To be displayed in the AWS Management Console, the data must be encoded in the H.264 codec, but Kinesis Video Streams can receive time-serialized data streams in any format.