Lumberyard
User Guide (Version 1.21)

The AWS Documentation website is getting a new look!
Try it now and let us know what you think. Switch to the new look >>

You can return to the original look by selecting English in the language selector above.

Video Playback

Component entity system is in preview release and is subject to change.

You can use the Video Playback component to play a video file on an entity in your Lumberyard level. For example, you can use a flat or plane entity to simulate a movie screen. You add the Video Playback component to it and specify a video file to display. You can use Script Canvas or Lua scripting to trigger the video to play, pause, or stop, depending on player actions.

Prerequisites

To use the Video Playback component, you must do the following:

You can also set your video to play in visual stereo (not audio stereo). To test this feature, you must use a virtual reality head mounted display (HMD).

Note

Audio isn't currently supported with the Video Playback component, but you can trigger audio playback separately if you want to play audio with your video.

Setting Up Video Playback

To set up video playback in Lumberyard, you must install either FFmpeg or LibAV.

  • If both are installed, Lumberyard uses FFmpeg.

  • To use LibAV, remove FFmpeg and see Installing LibAV.

Note

Certain third-party software might require a license. Consult the terms of service before installing the software.

Installing FFmpeg

To install FFmpeg for Lumberyard, follow these steps.

To install FFmpeg

  1. Go to FFmpeg Builds.

  2. Download the Shared and Dev versions of FFmpeg and unzip the files.

  3. Navigate to lumberyard_version/3rdParty and create a directory named FFmpeg.

  4. In FFmpeg, create a directory named 3.2.

    Note

    You must name the directory 3.2, regardless of the FFmpeg version that you're using.

  5. From the Dev version of FFmpeg, copy the include and lib folders to the lumberyard_version/3rdParty/FFmpeg/3.2 directory.

  6. From the Shared version of FFmpeg, copy the bin folder to the lumberyard_version/3rdParty/FFmpeg/3.2 directory.

  7. In the 3.2 directory, verify that you have the following folders:

    • bin

    • include

    • lib

  8. Run Lumberyard Setup Assistant and, on the Install optional SDKs page, verify that Lumberyard detects FFmpeg.

    
                            Install FFmpeg for Lumberyard.

Installing LibAV

To install LibAV for Lumberyard, follow these steps.

To install LibAV

  1. Go to http://builds.libav.org/windows/.

  2. Select the release-lgpl build and download LibAV. As of this writing, version 11.7 is the latest build.

  3. Navigate to lumberyard_version/3rdParty and create a directory named libav.

  4. In the libav directory, create a directory named 11.7.

    Note

    You must name the directory 11.7, regardless of the LibAV version that you're using.

  5. Extract the .7z file to the 11.7 directory.

    Note

    To open and extract .7z files, you must use a 7z application, such as 7-Zip.

    In the 11.7 directory, you should have the following:

    • A directory named usr

    • config.log

    • md5sum

  6. Run Lumberyard Setup Assistant and on the Install optional SDKs page, verify that Lumberyard detects LibAV.

    
                            Install LibAV for Lumberyard.

Using the Video Playback Component

After you complete the Prerequisites, you can use the Video Playback component.

Video playback supports the following container formats:

  • .mp4

  • .mkv (recommended)

  • .webm (recommended)

Video playback supports the following codecs:

  • h.264

  • h.265

  • VP8 (recommended)

  • VP9 (recommended)

The basic setup for the Video Playback component includes the following:

  • Add a Camera component

  • Add a Mesh and Video Playback component

  • Configure your material

To use the Video Playback component

  1. If you don't yet have a camera in your scene, place a Camera component where your video playback is to be placed.

    You can use the camera to view your video playback. Ensure that the camera is facing the direction where you place your video playback component.

  2. Create an entity. For more information, see Creating an Entity.

  3. Use the Entity Inspector to add a Mesh component to your entity.

  4. For the Mesh component, select a Mesh asset. This is the asset that your video renders on. A cube or plane is a good test mesh.

    
                        Mesh component properties in
                            Lumberyard Editor
  5. Add the Video Playback component to the same entity.

  6. In the Video Playback component, for Video, select the video to display.

  7. For Texture name, enter dollar sign ($) and a name for your texture. You can enter any name, but it must begin with a $ character to indicate that it's a render target. For example, $videotest is a valid name, but videotest isn't.

    
                        Video Playback component properties in
                            Lumberyard Editor
  8. For Frame queue ahead count, set the number of frames to buffer.

    We recommend that you use a value from 1 to 3.

    Queueing too many frames to buffer (for example, a value of 100 frames) can use too much memory and cause performance issues.

  9. Open the Material Editor.

  10. To create a material. click theAdd New Item icon. Enter a descriptive name, such as myvideomaterial.

    
                        Video Playback component material.
  11. Under Texture Maps, on the Diffuse line, enter the name of your video component's Texture name field. You must include the $ character.

    
                        Diffuse property for the texture name.
  12. Close the Material Editor and return to the Entity Inspector. In the Mesh component, for the Material override property, select the material that you created.

    
                        Select the material override in the Mesh
                            component.

You can trigger the video to play at the start of your game using either flow graphs or Lua scripting. The following procedure shows you how to create a simple flow graph to start playing the video when your game starts.

This topic references tools and features that are legacy. If you want to use legacy tools in Lumberyard Editor, disable the CryEntity Removal gem using the Project Configurator or the command line. To learn more about legacy features, see the Amazon Lumberyard Legacy Reference.

To set up a flow graph to start your video

  1. In the viewport, right-click on the static mesh/video playback entity. Then click Flowgraph, Add. Type a name for your new flow graph.

  2. Drag a Game:Start node and a VideoPlayback:Play node onto your flow graph. Connect the output port of the Game:Start node to the Activate port of the VideoPlayback:Play node.

    Right-click Choose Entity in the VideoPlayback:Play node and click Assign graph entity.

  3. To play the game and test your video playback, press Ctrl G.

    Note

    Audio playback is not supported with this component. You can trigger audio playback separately.

Setting Up Stereo Video Playback

Before setting up stereo video playback, ensure that you have completed the setup instructions in Setting Up Video Playback.

Stereo video playback means that the video presents a slightly different image for each eye, creating a 3D feel. Stereo in this case refers to visual stereo only; audio is not supported in the video playback component. While audio is not supported and must be synced externally, it is possible to play back spatialized audio. To do so, you must use the full, commercial version of Wwise and a 3D spatializer plugin such as Oculus Spatializer or RealSpace 3D.

To set up stereo video playback, your source video file must be laid out for stereo. Lumberyard supports videos in a top-bottom or bottom-top layout. You must have a VR headset to verify that the video is playing in stereo.

To set up stereo video playback, follow the instructions in Using the Video Playback Component. The only differences in the setup are the following:

  • You must use a source video file with 3D or stereo information

  • Set the Stereo Layout property initially to Auto-Detect. If it fails to auto-detect, manually set it to Top-Bottom or Bottom-Top.

    All supported video files should have their stereo layout written into their metadata. This, however, is not a requirement and may not have been inserted by your encoder. If you would like to inject stereo metadata into your video, see https://support.google.com/jump/answer/7044297?hl=en.

When you enter game mode (using Ctrl G), you should see the left eye of your video play. If you do not see this, try changing your Stereo layout setting.

To verify that your video is playing in stereo, you must enter VR mode. You can enter VR mode by clicking VR Preview at the bottom right corner of the viewport. Then press Ctrl G to enter game mode. If your VR Preview button isn’t enabled, or you can’t get into VR preview mode, ensure that your VR headset is working outside of Lumberyard and then restart the Lumberyard editor.

Playing stereo video is resource intensive. Because the video is often close in proximity to the player, it becomes easy to detect inconsistencies and artifacts in the video. To prevent that, use higher resolution videos whenever possible. To conserve resources, do not play more than one or two high resolution stereo videos at a time.

Lua Bindings for Video Playback

You can use Lua bindings to interact programmatically with video playback components that you’ve placed in your scene. Lua provides a way to establish complex logic for playing, pausing, and stopping videos.

Global Functions

The following functions provide programming interfaces for the video playback systems.

VideoPlaybackRequestBusSender

Parameters

EntityID

Return

Returns the VideoPlaybackRequestBusSender object that is connected to the specified entity. For more information, see VideoPlaybackRequestBusSender Object.

VideoPlaybackNotificationBusHandler

Exposes callbacks to your Lua script that are triggered by events during video playback.

For more information, see VideoPlaybackNotificationBusHandler Object.

Parameters

Table – The Lua table to which you want to expose the callback functions. Pass self to expose the callbacks to the current Lua script.

EntityId

Return

Returns the VideoPlaybackRequestBusSender object that is connected to the specified entity. For more information, see VideoPlaybackRequestBusSender Object.

VideoPlaybackRequestBusSender Object

The VideoPlaybackRequestBusSender object contains functions with which you can send requests to the video playback component.

Bool IsPlaying()

Returns true if the video is playing. If the video is paused or stopped, returns false.

Void Play()

Plays the video. If no video is selected or the video is already playing, this has no effect.

Void Pause()

Pauses the video. If the video is already paused, this has no effect.

Void Stop()

Stops the video and remains on the last frame. When the video plays again, it begins at the first frame of the video. If the video is already stopped, this has no effect. If the video is playing or paused, the video stops.

Void EnableLooping(Bool)

Sets whether this video automatically restarts from the beginning once the end of the video is reached. Pass true to enable looping or false to disable looping. Looping is disabled by default.

Void SetPlaybackSpeed(Float)

Sets how fast the video plays. For example, 1.0 is normal speed, 0.5 is half speed, 2.0 is double speed, and so on.

Caution is advised when setting the video speed. Setting a speed that is too high can result in choppy playback.

VideoPlaybackNotificationBusHandler Object

The VideoPlaybackNotificationBusHandler object exposes callback functions to your Lua script that are triggered by events that happen during video playback.

Void OnPlaybackStarted()

Called when video playback begins.

Void OnPlaybackPaused()

Called when video playback pauses. Not called when video stops.

Void OnPlaybackStopped()

Called when video playback is stopped by the user. If the video reaches the end and is not set to loop, this function is not called.

Void OnPlaybackFinished()

Called when all frames in the video are played. This is not called if the user manually stops video playback. If looping is enabled, this function is called every time the video loops.

Setting Up Video Playback with Flow Graph

Using the Video Playback flow graph nodes, you can set up video playback actions that trigger based on certain actions.

In the following flow graph example, the PlayPauseMovie input event (1) plays or pauses the video on the attached graph entity. The StopMovie input event (2) stops the movie.

The PlayPauseMovie and StopMovie events each connect to an IsPlaying node (3), which checks whether the video should be played, paused, or stopped.

The Play node (4) sets PlaybackSpeed to 1 (normal speed) and enables the Loop setting (Boolean value of 1), indicating that video looping is enabled. The Play node (4) also triggers the video to play on Game:Start (5).

The PlaybackEvents node (6) triggers the Debug nodes (7) to print to the console whenever the video plays, pauses, stops, and finishes.