Broadcasting to Amazon IVS: SDK for iOS Guide - Amazon Interactive Video Service

Broadcasting to Amazon IVS: SDK for iOS Guide

The Amazon Interactive Video Service (IVS) iOS broadcast SDK provides the interfaces required to broadcast to Amazon IVS on iOS.

The AmazonIVSBroadcast module implements the interface described in this document. The following operations are supported:

  • Set up (initialize) a broadcast session.

  • Manage broadcasting.

  • Attach and detach input devices.

  • Manage a composition session.

  • Receive events.

  • Receive errors.

Latest version of iOS broadcast SDK: 1.0.0 (Release Notes)

Reference documentation: For information on the most important methods available in the Amazon IVS iOS broadcast SDK, see the reference documentation at https://aws.github.io/amazon-ivs-broadcast-docs/1.0.0/ios/.

Sample code: See the iOS sample repository on GitHub: https://github.com/aws-samples/amazon-ivs-broadcast-ios-sample.

Platform requirements: iOS 11 or greater.

Getting Started

Install the Library

We recommend that you integrate the player SDK via CocoaPods. (Alternatively, you can manually add the framework to your project.)

Recommended: Integrate the Broadcast SDK (CocoaPods)

Releases are published via CocoaPods under the name AmazonIVSBroadcast. Add this dependency to your Podfile:

pod 'AmazonIVSBroadcast'

Run pod install and the SDK will be available in your .xcworkspace.

Alternate Approach: Install the Framework Manually

  1. Download the latest version from https://broadcast.live-video.net/1.0.0/AmazonIVSBroadcast.xcframework.zip.

  2. Extract the contents of the archive. AmazonIVSBroadcast.xcframework contains the SDK for both device and simulator.

  3. Embed AmazonIVSBroadcast.xcframework by dragging it into the Frameworks, Libraries, and Embedded Content section of the General tab for your application target.

    
                                The Frameworks, Libraries, and
                                        Embedded Content section of the General tab for your application
                                    target.

Implement IVSBroadcastSession.Delegate

Implement IVSBroadcastSession.Delegate, which allows you to receive state updates and device-change notifications:

extension ViewController : IVSBroadcastSession.Delegate { func broadcastSession(_ session: IVSBroadcastSession, didChange state: IVSBroadcastSession.State) { print("IVSBroadcastSession did change state \(state)") } func broadcastSession(_ session: IVSBroadcastSession, didEmitError error: Error) { print("IVSBroadcastSession did emit error \(error)") } }

Request Permissions

Your app must request permission to access the user’s camera and mic. (This is not specific to Amazon IVS; it is required for any application that needs access to cameras and microphones.)

Here, we check whether the user has already granted permissions and, if not, we ask for them:

switch AVCaptureDevice.authorizationStatus(for: .video) { case .authorized: // permission already granted. case .notDetermined: AVCaptureDevice.requestAccess(for: .video) { granted in // permission granted based on granted bool. } case .denied, .restricted: // permission denied. @unknown default: // permissions unknown. }

You need to do this for both .video and .audio media types, if you want access to cameras and microphones, respectively.

You also need to add entries for NSCameraUsageDescription and NSMicrophoneUsageDescription to your Info.plist. Otherwise, your app will crash when trying to request permissions.

Disable the Application Idle Timer

This is optional but recommended. It prevents your device from going to sleep while using the broadcast SDK, which would interrupt the broadcast.

override func viewDidAppear(_ animated: Bool) { super.viewDidAppear(animated) UIApplication.shared.isIdleTimerDisabled = true } override func viewDidDisappear(_ animated: Bool) { super.viewDidDisappear(animated) UIApplication.shared.isIdleTimerDisabled = false }

(Optional) Set Up AVAudioSession

By default, the broadcast SDK will set up your application’s AVAudioSession. If you want to manage this yourself, set this to noAction: IVSBroadcastSession.applicationAudioSessionStrategy

If you are manually setting up your AVAudioSession, at a minimum you need to set the category as .record or .playbackAndRecord, and set it to active. If you want to record audio from Bluetooth devices, you need to specify the .allowBluetooth option as well:

do { try AVAudioSession.sharedInstance().setCategory(.record, options: .allowBluetooth) try AVAudioSession.sharedInstance().setActive(true) } catch { print("Error configuring AVAudioSession") }

We recommend that you let the SDK handle this for you. Otherwise, if you want to choose between different audio devices, you will need to manually manage the ports.

Create the Broadcast Session

The broadcast interface is IVSBroadcastSession. Initialize it as shown below:

let broadcastSession = try IVSBroadcastSession( configuration: IVSPresets.configurations().standardLandscape(), descriptors: IVSPresets.devices().frontCamera(), delegate: self)

Also see Create the Broadcast Session (Advanced Version)

Set the IVSImagePreviewView for Preview

If you want to display a preview for an active camera device, add the preview IVSImagePreviewView for the device to your view hierarchy:

// If the session was just created, execute the following // code in the callback of IVSBroadcastSession.awaitDeviceChanges // to ensure all devices have been attached. if let devicePreview = try broadcastSession.listAttachedDevices() .compactMap({ $0 as? IVSImageDevice }) .first? .previewView() { previewView.addSubview(devicePreview) }

Start a Broadcast

The hostname that you receive in the ingestEndpoint response field of the GetChannel endpoint needs to have rtmps:// prepended and /app appended. The complete URL should be in this format: rtmps://{{ ingestEndpoint }}/app

try broadcastSession.start(with: IVS_RTMPS_URL, streamKey: IVS_STREAMKEY)

Stop a Broadcast

broadcastSession.stop()

Manage Lifecycle Events

Audio Interruptions

There are several scenarios where the broadcast SDK will not have exclusive access to audio-input hardware. Some example scenarios that you need to handle are:

  • User receives a phone call or FaceTime call

  • User activates Siri

Apple makes it easy to respond to these events by subscribing to AVAudioSession.interruptionNotification:

NotificationCenter.default.addObserver( self, selector: #selector(audioSessionInterrupted(_:)), name: AVAudioSession.interruptionNotification, object: nil)

Then you can handle the event with something like this:

// This assumes you have a variable `isRunning` which tracks if the broadcast is currently live, and another variable `wasRunningBeforeInterruption` which tracks whether the broadcast was active before this interruption to determine if it should resume after the interruption has ended. @objc private func audioSessionInterrupted(_ notification: Notification) { guard let userInfo = notification.userInfo, let typeValue = userInfo[AVAudioSessionInterruptionTypeKey] as? UInt, let type = AVAudioSession.InterruptionType(rawValue: typeValue) else { return } switch type { case .began: wasRunningBeforeInterruption = isRunning if isRunning { broadcastSession.stop() } case .ended: defer { wasRunningBeforeInterruption = false } guard let optionsValue = userInfo[AVAudioSessionInterruptionOptionKey] as? UInt else { return } let options = AVAudioSession.InterruptionOptions(rawValue: optionsValue) if options.contains(.shouldResume) && wasRunningBeforeInterruption { try broadcastSession.start( with: IVS_RTMPS_URL, streamKey: IVS_STREAMKEY) } @unknown default: break } }

App Going Into Background

Standard applications on iOS are not allowed to use cameras in the background. There also are restrictions on video encoding in the background: since hardware encoders are limited, only foreground applications have access. Because of this, the broadcast SDK automatically terminates its session and sets its isReady property to false. When your application is about to enter the foreground again, the broadcast SDK reattaches all the devices to their original IVSMixerSlotConfiguration entries.

The broadcast SDK does this by responding to UIApplication.didEnterBackgroundNotification and UIApplication.willEnterForegroundNotification.

If you are providing custom image sources, you should be prepared to handle these notifications. You may need to take extra steps to tear them down before the stream is terminated.

Media Services Lost

In very rare cases, the entire media subsystem on an iOS device will crash. In this scenario, we can no longer broadcast. It is up to your application to respond to these notifications appropriately. At a minimum, subscribe to these notifications:

  • mediaServicesWereLostNotification — Respond by stopping your broadcast and completely deallocating your IVSBroadcastSession . All internal components used by the broadcast session will be invalidated.

  • mediaServicesWereResetNotification — Respond by notifying your users that they can broadcast again. Depending on your use case, you may be able to automatically start broadcasting again at this point.

Advanced Use Cases

Here we present some advanced use cases. Start with the basic setup above and continue here.

Create a Broadcast Configuration

Here we create a custom configuration with two mixer slots that allow us to bind two video sources to the mixer. One (custom) is full screen and laid out behind the other (camera), which is smaller and in the bottom-right corner. Note that for the custom slot we do not set a position, size, or aspect mode. Because we do not set these parameters, the slot uses the video settings for size and position.

let config = IVSBroadcastConfiguration() try config.audio.setBitrate(128_000) try config.video.setMaxBitrate(3_500_000) try config.video.setMinBitrate(500_000) try config.video.setInitialBitrate(1_500_000) try config.video.setSize(CGSize(width: 1280, height: 720)) config.video.defaultAspectMode = .fit config.mixer.slots = [ try { let slot = IVSMixerSlotConfiguration() // Do not automatically bind to a source slot.preferredAudioInput = .unknown // Bind to user image if unbound slot.preferredVideoInput = .userImage try slot.setName("custom") return slot }(), try { let slot = IVSMixerSlotConfiguration() slot.zIndex = 1 slot.aspect = .fill slot.size = CGSize(width: 300, height: 300) slot.position = CGPoint(x: config.video.size.width - 200, y: config.video.size.height - 200) try slot.setName("camera") return slot }() ]

Create the Broadcast Session (Advanced Version)

Create an IVSBroadcastSession as you did in the basic example, but provide your custom configuration here. Also provide nil for the device array, as we will add those manually.

let broadcastSession = try IVSBroadcastSession( configuration: config, // The configuration we created above descriptors: nil, // We’ll manually attach devices after delegate: self)

Iterate and Attach a Camera Device

Here we iterate through input devices that the SDK has detected. The SDK will only return built-in devices on iOS. Even if Bluetooth audio devices are connected, they will appear as a built-in device. For more information, see Known Issues and Workarounds.

Once we find a device that we want to use, we call attachDevice to attach it:

let frontCamera = IVSBroadcastSession.listAvailableDevices() .filter { $0.type == .camera && $0.position == .front } .first if let camera = frontCamera { broadcastSession.attach(camera, toSlotWithName: "camera") { device, error in // check error } }

Swap Cameras

// This assumes you’ve kept a reference called `currentCamera` that points to the current camera. let wants: IVSDevicePosition = (currentCamera.descriptor().position == .front) ? .back : .front // Remove the current preview view since the device will be changing. previewView.subviews.forEach { $0.removeFromSuperview() } let foundCamera = IVSBroadcastSession .listAvailableDevices() .first { $0.type == .camera && $0.position == wants } guard let newCamera = foundCamera else { return } broadcastSession.exchangeOldDevice(currentCamera, withNewDevice: newCamera) { newDevice, _ in currentCamera = newDevice if let camera = newDevice as? IVSImageDevice { do { previewView.addSubview(try finalCamera.previewView()) } catch { print("Error creating preview view \(error)") } } }

Create a Custom Input Source

To input sound or image data that your app generates, use createImageSource or createAudioSource. Both these methods create virtual devices that can be bound to the mixer like any other device.

The devices returned by both these methods accept a CMSampleBuffer through its onSampleBuffer function:

  • For video sources, the pixel format must be kCVPixelFormatType_32BGRA, 420YpCbCr8BiPlanarFullRange, or 420YpCbCr8BiPlanarVideoRange.

  • For audio sources, the buffer must contain Linear PCM data.

You cannot use an AVCaptureSession with camera input to feed a custom image source while also using a camera device provided by the broadcast SDK. If you want to use multiple cameras simultaneously, use AVCaptureMultiCamSession and provide two custom image sources.

Custom image sources primarily should be used with static content such as images, or with video content:

let customImageSource = broadcastSession.createImageSource(withName: "video") try broadcastSession.attach(customImageSource, toSlotWithName: "custom")

Monitor Network Connectivity

It is common for mobile devices to temporarily lose and regain network connectivity while on the go. Because of this, it is important to monitor your app’s network connectivity and respond appropriately when things change.

When the broadcaster's connection is lost, the broadcast SDK's state will change to error and then disconnected. You will be notified of these changes through the IVSBroadcastSessionDelegate. When you receive these state changes:

  1. Monitor your broadcast app’s connectivity state and call start with your endpoint and stream key, once your connection has been restored.

  2. Important: Monitor the state delegate callback and ensure that the state changes to connected after calling start again.

Detach a Device

If you want to detach and not replace a device, detach it with IVSDevice or IVSDeviceDescriptor:

broadcastSession.detachDevice(currentCamera)

ReplayKit Integration

To stream the device’s screen and system audio on iOS, you must integrate with ReplayKit. The Amazon IVS Player SDK makes it easy to integrate ReplayKit using IVSReplayKitBroadcastSession. In your RPBroadcastSampleHandler subclass, create an instance of IVSReplayKitBroadcastSession, then:

  • Start the session in broadcastStarted

  • Stop the session in broadcastFinished

The session object will have three custom sources for screen images, app audio, and microphone audio. Pass the CMSampleBuffers provided in processSampleBuffer to those custom sources.

To handle device orientation, you need to extract ReplayKit-specific metadata from the sample buffer. Use the following code:

let imageSource = session.systemImageSource; if #available(iOSApplicationExtension 11.0, *) { if let orientationAttachment = CMGetAttachment(sampleBuffer, key: RPVideoSampleOrientationKey as CFString, attachmentModeOut: nil) as? NSNumber, let orientation = CGImagePropertyOrientation(rawValue: orientationAttachment.uint32Value) { switch orientation { case .up, .upMirrored: imageSource.setHandsetRotation(0) case .down, .downMirrored: imageSource.setHandsetRotation(Float.pi) case .right, .rightMirrored: imageSource.setHandsetRotation(-(Float.pi / 2)) case .left, .leftMirrored: imageSource.setHandsetRotation((Float.pi / 2)) } } }

It is possible to integrate ReplayKit using IVSBroadcastSession instead of IVSReplayKitBroadcastSession. However, the ReplayKit-specific variant has several modifications to reduce the internal memory footprint, to stay within Apple’s memory ceiling for broadcast extensions.

To evaluate your user’s connection before starting a broadcast, use the recommendedVideoSettings method to run a brief test. As the test runs, you will receive several recommendations, ordered from most to least recommended. In this version of the SDK, it is not possible to reconfigure the current IVSBroadcastSession, so you must deallocate it and then create a new one with the recommended settings. You will continue to receive IVSBroadcastSessionTestResults until the result.status is Success or Error. You can check progress with result.progress.

Amazon IVS supports a maximum bitrate of 8.5 Mbps (for channels whose type is STANDARD), so the maximumBitrate returned by this method never exceeds 8.5 Mbps. To account for small fluctuations in network performance, the recommended initialBitrate returned by this method is slightly less than the true bitrate measured in the test. (Using 100% of the available bandwidth usually is inadvisable.)

func runBroadcastTest() { self.test = session.recommendedVideoSettings(with: IVS_RTMPS_URL, streamKey: IVS_STREAMKEY) { [weak self] result in if result.status == .success { this.recommendation = result.recommendations[0]; } } }

Known Issues and Workarounds

Please report all issues to Amazon IVS Support.

  • Changing Bluetooth audio routes can be unpredictable. If you connect a new device mid-session, iOS may or may not automatically change the input route. Also, it is not possible to choose between multiple Bluetooth headsets that are connected at the same time.

    Workaround: If you plan to use a Bluetooth headset, connect it before starting the broadcast and leave it connected throughout the broadcast.

  • A bug in ReplayKit causes rapid memory growth when plugging in a wired headset during a stream.

    Workaround: Start the stream with the wired headset already plugged in, use a Bluetooth headset, or do not use an external microphone.

  • If at any point during a ReplayKit stream you enable the microphone and then interrupt the audio session (e.g., with a phone call or by activating Siri), system audio will stop working. This is a ReplayKit bug that we are working with Apple to resolve.

    Workaround: On an audio interruption, stop the broadcast and alert the user.

  • AirPods do not record any audio if the AVAudioSession category is set to record. By default, the SDK uses playAndRecord, so this issue manifests only if the category is changed to record.

    Workaround: If there is a chance that AirPods will be used to record audio, use playAndRecord even if your application is not playing back media.

  • Submitting audio data faster than realtime (using a custom audio source) results in audio drift.

    Workaround: Do not submit audio data faster than realtime.