Server-Side Composition (Real-Time Streaming)

Server-side composition uses an IVS server to mix audio and video from all stage participants and then sends this mixed video to an IVS channel (e.g., to reach a larger audience) or an S3 bucket. Server-side composition is invoked through IVS control-plane endpoints in the stage’s home region.

Broadcasting or recording a stage using server-side composition offers numerous benefits, making it an attractive choice for users seeking efficient and reliable cloud-based video workflows.

This diagram illustrates how server-side composition works:

Benefits

Compared to client-side composition, server-side composition has the following benefits:

• Reduced client load — With server-side composition, the burden of processing and combining audio and video sources is shifted from individual client devices to the server itself. Server-side composition eliminates the need for client devices to use their CPU and network resources for compositing the view and transmitting it to IVS. This means viewers can watch the broadcast without their devices having to handle resource-intensive tasks, which can lead to improved battery life and smoother viewing experiences.

• Consistent quality — Server-side composition allows for precise control over the quality, resolution, and bitrate of the final stream. This ensures a consistent viewing experience for all viewers, regardless of their individual devices' capabilities.

• Resilience — By centralizing the composition process on the server, the broadcast becomes more robust. Even if a publisher device experiences technical limitations or fluctuations, the server can adapt and provide a smoother stream to all audience members.

• Bandwidth efficiency — Since the server handles the composition, stage publishers do not have to spend extra bandwidth broadcasting the video to IVS.

Alternatively, to broadcast a stage to an IVS channel, you can do the composition client side; see Enabling Multiple Hosts on an IVS Stream in the IVS Low-Latency Streaming User Guide.

IVS API

Server-side composition uses these key API elements:

• An EncoderConfiguration object allows you to customize the format of the video to be generated (height, width, bitrate, and other streaming parameters). You can reuse an EncoderConfiguration every time you call the StartComposition endpoint.

• Composition endpoints track the video composition and output to an IVS channel.

• StorageConfiguration tracks the S3 bucket where compositions are recorded.

To use server-side composition, you need to create an EncoderConfiguration and attach it when calling the StartComposition endpoint. In this example, the SquareVideo EncoderConfiguration is used in two Compositions:

For complete information, see IVS Real-Time Streaming API Reference.

Layouts

The StartComposition endpoint offers two layout options: grid and pip (Picture-in-Picture).

Grid Layout

The grid layout arranges stage participants in a grid of equally sized slots. It provides several customizable properties:

• videoAspectRatio sets the participant display mode to control the aspect ratio of video tiles.

• videoFillMode defines how video content fits within the participant tile.

• gridGap specifies the spacing between participant tiles in pixels.

• omitStoppedVideo allows excluding stopped video streams from the composition.

• featuredParticipantAttribute identifies the featured slot. When this is set, the featured participant is displayed in a larger slot on the main screen, with other participants shown below it.

For details on grid layout (including valid values and defaults for all fields), see the GridConfiguration data type.

Picture-in-Picture (PiP) Layout

The PiP layout enables displaying a participant in an overlay window with configurable size, position, and behavior. Key properties include:

• pipParticipantAttribute specifies the participant for the PiP window.

• pipPosition determines the corner position of the PiP window.

• pipWidth and pipHeightconfigure the width and height of the PiP window.

• pipOffset sets the offset position of the PiP window in pixels from the closest edges.

• pipBehavior defines PiP behavior when all other participants have left.

Like the grid layout, the PiP supports featuredParticipantAttribute, omitStoppedVideo, videoFillMode, and gridGap to further customize the composition.

For details on PiP layout (including valid values and defaults for all fields), see the PipConfiguration data type.

Note: The maximum resolution supported by a stage publisher on server-side composition is 1080p. If a publisher sends video higher than 1080p, the publisher will be rendered as an audio-only participant.

Important: Ensure your application does not depend on the specific features of the current layout, such as size and position of tiles. Visual improvements to layouts can be introduced at any time.

Getting Started

Prerequisites

To use server-side composition, you must have a stage with active publishers and use an IVS channel and/or an S3 bucket as the composition destination. Below, we describe one possible workflow that uses EventBridge events to start a composition that broadcasts the stage to an IVS channel when a participant publishes. Alternatively, you can start and stop compositions based on your own app logic. See Composite Recording for another example which showcases the use of server-side composition to record a stage directly to an S3 bucket.

1. Create an IVS channel. See Getting Started with Amazon IVS Low-Latency Streaming.

2. Create an IVS stage and participant tokens for each publisher.

3. Create an EncoderConfiguration.

4. Join the stage and publish to it. (See the "Publishing and Subscribing" sections of the real-time streaming broadcast SDK guides: Web, Android, and iOS.)

5. When you receive a Participant Published EventBridge event, call StartComposition with your desired layout configuration.

6. Wait for a few seconds and see the composited view in the channel playback.

Note: A Composition performs auto-shutdown after 60 seconds of inactivity from publisher participants on the stage. At that point, the Composition is terminated and transitions to a STOPPED state. A Composition is automatically deleted after a few minutes in the STOPPED state.

CLI Instructions

Using the AWS CLI is an advanced option and requires that you first download and configure the CLI on your machine. For details, see the AWS Command Line Interface User Guide.

Now you can use the CLI to create and manage resources. The Composition endpoints are under the ivs-realtime namespace.

Create the EncoderConfiguration Resource

An EncoderConfiguration is an object that allows you to customize the format of the generated video (height, width, bitrate, and other streaming parameters). You can reuse an EncoderConfiguration every time you call the Composition endpoint, as explained in the next step.

The command below creates an EncoderConfiguration resource that configures server-side video composition parameters like video bitrate, frame rate and resolution:

aws ivs-realtime create-encoder-configuration --name "MyEncoderConfig" --video "bitrate=2500000,height=720,width=1280,framerate=30"

The response is:

{
   "encoderConfiguration": {
      "arn": "arn:aws:ivs:us-east-1:927810967299:encoder-configuration/9W59OBY2M8s4",
      "name": "MyEncoderConfig",
      "tags": {},
      "video": {
         "bitrate": 2500000,
         "framerate": 30,
         "height": 720,
         "width": 1280
      }
   }
}

Start a Composition

Using the EncoderConfiguration ARN provided in the response above, create your Composition resource:

Grid Layout Example

aws ivs-realtime start-composition --stage-arn "arn:aws:ivs:us-east-1:927810967299:stage/8faHz1SQp0ik" --destinations '[{"channel": {"channelArn": "arn:aws:ivs:us-east-1:927810967299:channel/DOlMW4dfMR8r", "encoderConfigurationArn": "arn:aws:ivs:us-east-1:927810967299:encoder-configuration/9W59OBY2M8s4"}}]' --layout '{"grid":{"featuredParticipantAttribute":"isFeatured","videoFillMode":"COVER","gridGap":0}}'

PiP Layout Example

aws ivs-realtime start-composition --stage-arn "arn:aws:ivs:us-east-1:927810967299:stage/8faHz1SQp0ik" --destinations '[{"channel": {"channelArn": "arn:aws:ivs:us-east-1:927810967299:channel/DOlMW4dfMR8r", "encoderConfigurationArn": "arn:aws:ivs:us-east-1:927810967299:encoder-configuration/DEkQHWPVaOwO"}}]' --layout '{"pip":{"pipParticipantAttribute":"isPip","pipOffset":10,"pipPosition":"TOP_RIGHT"}}'

Note: You can use this tool to more easily generate the --layout configuration based on your layout choices.

The response will show that the Composition is created with a STARTING state. Once the Composition starts publishing the composition, the state transitions to ACTIVE. (You can see the state by calling the ListCompositions or GetComposition endpoint.)

Once a Composition is ACTIVE, the composite view of the IVS stage is visible on the IVS channel, using ListCompositions:

aws ivs-realtime list-compositions

The response is:

{
   "compositions": [
      {
         "arn": "arn:aws:ivs:us-east-1:927810967299:composition/YVoaXkKdEdRP",
         "destinations": [
            {
               "id": "bD9rRoN91fHU",
               "startTime": "2023-09-21T15:38:39+00:00",
               "state": "ACTIVE"
            }
         ],
         "stageArn": "arn:aws:ivs:us-east-1:927810967299:stage/8faHz1SQp0ik",
         "startTime": "2023-09-21T15:38:37+00:00",
         "state": "ACTIVE",
         "tags": {}
      }
   ]
}

Note: You need to have publisher participants actively publishing to the stage to keep the composition alive. For more information, see the "Publishing and Subscribing" sections of the real-time streaming broadcast SDK guides: Web, Android, and iOS. You must create a distinct stage token for each participant.

Enable Screen Share

To use a fixed screen-share layout, follow the steps below.

Create the EncoderConfiguration Resource

The command below creates an EncoderConfiguration resource that configures server-side composition parameters (video bitrate, framerate, and resolution).

aws ivs-realtime create-encoder-configuration --name "test-ssc-with-screen-share" --video={bitrate=2000000,framerate=30,height=720,width=1280}

Create a stage participant token with a screen-share attribute. Since we will specify screen-share as the name of the featured slot, we need to create a stage token with the screen-share attribute set to true:

aws ivs-realtime create-participant-token --stage-arn "arn:aws:ivs:us-east-1:123456789012:stage/u9OiE29bT7Xp" --attributes screen-share=true

The response is:

{
   "participantToken": {
      "attributes": {
         "screen-share": "true"
      },
      "expirationTime": "2023-08-04T05:26:11+00:00",
      "participantId": "E813MFklPWLF",
      "token": "eyJhbGciOiJLTVMiLCJ0eXAiOiJKV1QifQ.eyJleHAiOjE2OTExMjY3NzEsImlhdCI6MTY5MTA4MzU3MSwianRpIjoiRTgxM01Ga2xQV0xGIiwicmVzb3VyY2UiOiJhcm46YXdzOml2czp1cy1lYXN0LTE6OTI3ODEwOTY3Mjk5OnN0YWdlL3U5T2lFMjliVDdYcCIsInRvcGljIjoidTlPaUUyOWJUN1hwIiwiZXZlbnRzX3VybCI6IndzczovL3VzLWVhc3QtMS5ldmVudHMubGl2ZS12aWRlby5uZXQiLCJ3aGlwX3VybCI6Imh0dHBzOi8vYjJlYTVjMmZmMzU1Lmdsb2JhbC53aGlwLmxpdmUtdmlkZW8ubmV0IiwiYXR0cmlidXRlcyI6eyJzY3JlZW4tc2hhcmUiOiJ0cnVlIn0sImNhcGFiaWxpdGllcyI6eyJhbGxvd19wdWJsaXNoIjp0cnVlLCJhbGxvd19zdWJzY3JpYmUiOnRydWV9LCJ2ZXJzaW9uIjoiMC4zIn0.MGUCMFvMzv35O4yVzM9tIWZl7n3mmFQhleqsRSBx_G2qT2YUDlWSNg6H1vL7sAWQMeydSAIxAIvdfqt3Fh1MLiyelc9NnTjI5hL3YPKqDX6J3NDH1fksh8_5y1jztoPDy4yVA5OmtA"
   }
}

Start the Composition

To start the composition using the screen-share feature, we use this command:

aws ivs-realtime start-composition --stage-arn "arn:aws:ivs:us-east-1:927810967299:stage/8faHz1SQp0ik" --destinations  '[{"channel": {"channelArn": "arn:aws:ivs:us-east-1:927810967299:channel/DOlMW4dfMR8r", "encoderConfigurationArn": "arn:aws:ivs:us-east-1:927810967299:encoder-configuration/DEkQHWPVaOwO"}}]' --layout '{"grid":{"featuredParticipantAttribute":"screen-share"}}'

The response is:

{
   "composition" : {
      "arn" : "arn:aws:ivs:us-east-1:927810967299:composition/B19tQcXRgtoz",
      "destinations" : [ {
         "configuration" : {
            "channel" : {
               "channelArn" : "arn:aws:ivs:us-east-1:927810967299:channel/DOlMW4dfMR8r",
               "encoderConfigurationArn" : "arn:aws:ivs:us-east-1:927810967299:encoder-configuration/DEkQHWPVaOwO"
            },
            "name" : ""
         },
         "id" : "SGmgBXTULuXv",
         "state" : "STARTING"
      } ],
      "layout" : {
         "grid" : {
            "featuredParticipantAttribute" : "screen-share",
            "gridGap": 2,
            "omitStoppedVideo": false,
            "videoAspectRatio": "VIDEO"
         }
      },
      "stageArn" : "arn:aws:ivs:us-east-1:927810967299:stage/8faHz1SQp0ik",
      "startTime" : "2023-09-27T21:32:38Z",
      "state" : "STARTING",
      "tags" : { }
   }
}

When the stage participant E813MFklPWLF joins the stage, that participant’s video will be displayed in the featured slot, and all other stage publishers will be rendered below the slot:

Stop the Composition

To stop a composition at any point, call the StopComposition endpoint:

aws ivs-realtime stop-composition --arn arn:aws:ivs:us-east-1:927810967299:composition/B19tQcXRgtoz

Composition Lifecycle

Use the diagram below to understand the state transitions of a Composition. At a high level, the life cycle of a Composition is as follows:

1. A Composition resource is created when the user calls the StartComposition endpoint

2. Once IVS successfully starts the Composition, an “IVS Composition State Change (Session Start)” EventBridge event is sent. See Using EventBridge with IVS Real-Time Streaming for details about events.

3. Once a Composition is in an active state, the following can happen:

   • User stops the Composition — If the StopComposition endpoint is called, IVS initiates a graceful shutdown of the Composition, sending "Destination End" events followed by a "Session End" event.

   • Composition performs auto-shutdown — If no participant is actively publishing to the IVS stage, the Composition is finalized automatically after 60 seconds and EventBridge events are sent.

   • Destination failure — If a destination unexpectedly fails (e.g., the IVS channel gets deleted), the destination transitions to the RECONNECTING state and a “Destination Reconnecting” event is sent. If recovery is impossible, IVS transitions the destination to the FAILED state and a “Destination Failure” event is sent. IVS keeps the composition alive if at least one of its destinations is active.

4. Once the composition is in the STOPPED or FAILED state, it is automatically cleaned up after five minutes. (Then it no longer is retrieved by ListCompositions or GetComposition.)