Menu
Lumberyard
Developer Guide (Version 1.11)

Text-to-Speech Cloud Gem Portal

You can use the Cloud Gem Portal (CGP) to create text-to-speech (TTS) functionality for your game and import it into your project. This topic shows you how to perform the following tasks:

  • Add a character and specify a name, language, and voice for it

  • Add a speech line for the character

  • Add tags to speech lines and filter by tags

  • Create a file with multiple speech lines

  • Import the file into the Cloud Gem Portal for processing

  • Download generated speech files and previously generated packages

  • Use the lmbr_aws command to import the generated voice packages into your project

Prerequisites

This tutorial assumes that you have already performed the following steps to set up the Cloud Gem Portal (CGP) and the Text-to-Speech Gem:

  • You are using a Lumberyard Editor project that has the Text-to-Speech Gem enabled.

  • You have created a project stack and a deployment stack in Cloud Canvas Resource Manager that uses an Amazon Polly supported AWS region.

  • You have opened the Cloud Gem Portal. In Lumberyard Editor, choose AWS, Open Cloud Gem Portal.

If you don't meet the prerequisites, follow the steps in Setting Up and Accessing the Cloud Gem Portal to add Cloud Gem Text-to-Speech in the Project Configurator.

Using the Text to Speech Cloud Gem Portal

In the Cloud Gem Portal, on the Cloud Gems page, choose Text to Speech.


        Choose Text to Speech

The initial portal includes a Speech Library, Characters, and Generated Packages tabs. These are the only tabs that you need to get started. For information about the other tabs, see Configuring Runtime Capabilities and Caching and Text-to-Speech Troubleshooting Features.


        Text to Speech Cloud Gem initial screen

You can use the Text to Speech Cloud Gem Portal to create speech character content in a variety of ways. Use the workflow that suits you best. The following are some example workflows:

  • You create characters on the Characters tab with the voice and language characteristics that you want. Then use the Speech Library tab to create speech lines for the characters.

  • You create the speech lines that you want on the Speech Library tab. Then use the Characters tab to modify the default character that the cloud gem created for you for the speech lines.

  • You use the Speech Library tab to import speech lines from a .csv file that you created. The file can specify characters that you have already created in the portal, or you can let the cloud gem portal specify a default character for your speech lines.

Adding a Character

A text-to-speech character is defined by a name that you give the character, the language that the character speaks, and the male or female voice that the character uses. It can also have speech mark files that contain lip sync and closed captioning information.

To add a character

  1. On the Characters tab of the Cloud Gem Portal, choose New Character to add a character to the characters table.

    
              Choose New Character
  2. Fill in the following for the character.

    • Name – By default, the new character is named NewCharacter_<total number of rows in the characters table>. Provide a name for the character that has meaning in the context of your game.

    • Language – Choose the language that the character uses. You can choose from more than 20 languages. For a complete list of Amazon Polly languages and their corresponding voices, see Available Voices in the Amazon Polly Developer Guide.

    • Voice – Choose the name of the Amazon Polly male or female voice that you want for your character.

    • Speech Marks – Select Viseme to generate lip syncing information for the speech lines that you create. Select Sentence/Word to generate closed captioning information. These check boxes are selected by default.

    The next time that you add a character, the new character entry appears first in the list of characters. The values for a new character default to the values that you provided for the previous character.

  3. Choose Save.

  4. Choose Preview to hear the voice that you selected say, "Hi. My name is <name>."

  5. To alter the character's voice, choose the gear icon next to Preview. Then use the Advanced Settings dialog box to adjust the Volume, Rate, and Pitch of the voice.

    
              Advanced settings

Now you are ready to add a speech line to give the character something to say.

Adding a Speech Line

You can create speech lines one at a time in the Cloud Gem Portal.

To add an individual speech line

  1. In the Cloud Gem Portal, choose the Speech Library tab.

  2. Choose New Speech Line.

    
              Adding a speech line
  3. Fill in the following for the speech line.

    • Character – Choose the character that will speak the speech line. You can choose from the characters that you created on the Characters tab.

    • Speech Line – Replace the default NewSpeechLine_<the total number of rows in the table> with the text that you want to convert to speech.

    • Preview – Choose the Preview icon to hear an audio preview of your text. If you selected the Viseme option for the character that you are using, the facial rig performs lip synchronization as well.

    • Tags – You can use custom tags to categorize your speech lines. You can use these tags to filter the speech lines that you want to view in the Cloud Gem Portal. To add one or more tags, type the tag name and then press Enter after each tag. After you are finished adding tags for the speech line, choose Save.

  4. Choose Save.

The next time that you add a speech line, the new line appears first in the speech library. The Character and Tag values for the new speech line default to the values that you provided for the speech line that you last created.

Warning

We advise that you not store sensitive customer information in speech lines or character definitions.

Filtering Speech Lines

After you have created multiple speech lines that have tags, you can choose the Tags icon to filter them. Check the tags that you want to use for the filter, and then choose Apply.


            Filtering by tags

Importing Speech Lines from a .csv File

Instead of creating your speech lines one by one in the Cloud Gem Portal, you can prepare a .csv file that contains all of the lines that you require. Then you can use the Cloud Gem Portal to import the file.

Creating a Speech Line File

To create a speechline file in .csv format, see the following example. The example has rows that specify fields for character, speech line, time of day, mood, and stimulus.

Copy
Character,Speech Line,Time of Day,Mood,Stimulus Jane,Good Morning! How are you?,Morning,Happy, John,You stepped on my toe!,Afternoon,Angry,footSteppedOn Mary,Welcome to the Text to Speech Cloud Gem.,Night,Happy, Richard,Thank you for your help.,Afternoon,Happy,directionsReceived

The CGP treats the first line in the file as a header row. The second and following rows are data rows. Each data row should specify a character name and a line of speech for the character. A data row can have other fields that have meaning in the context of your game. You can map these fields to custom tags when you import the file.

Importing a Speech Line File

The following steps show how to import a speech line file.

To import a .csv speech line file

  1. On the Speech Library tab, choose Import .csv., and choose the .csv file to import.

  2. In the Import window, choose your custom mappings, and then choose Continue.

    
                Mapping fields
  3. Select the speech lines that you want to import, or select the check box at the top to import them all.

    
                Choosing lines to import
  4. Choose Continue. The imported lines appear in the speech library, where you can edit them further if you want.

    
                Imported speech lines

    If the character for a speech line does not exist, the cloud gem creates one based on the values of the last character that you created.

Downloading Speech Files

After you use the Cloud Gem Portal to create your speech line/character combinations in the AWS Cloud, you can download a .zip file package of the files that were generated. You can then import the voice package into your project assets.

To download speech files

  1. On the Speech Library tab of the Cloud Gem Portal, select the speech lines that you want to download. Then choose Download Speech Files (.zip).

    
              Downloading speech files
  2. Type a name for the .zip file to be downloaded, and then choose Download.

    
              File name for download

    To ensure uniqueness, a UUID is added to the file name that you specify. The final file name format is <file_name_you_specified><UUID>.zip.

  3. If your speech package is large, it might take some time to download. To check the status of a pending download, choose the cloud icon next to Download Speech Files (.zip).

    
              Pending downloads
  4. You can open the .zip file to view its contents. The file contains the voice (.pcm) files, the speech mark (.json) files, and a character_mapping.json file.

    
              Viewing downloaded contents

    Several speech lines can share a voice file if they use the same voice and speech line. They can also share a speech marks file if they are using the same voice, speech line, and speech marks.

Downloading Generated Packages

When you download a speech file, a copy is saved on the Generated Packages tab. This creates a repository of generated packages that you can download again if necessary.


          Downloading generated packages

To download one or more generated packages, select the packages that you want, and then choose Download Generated Packages.

Importing the Generated Voice Packages Into Your Project

After you have downloaded the voice packages that you generated, you can import them into your project. This removes the need for the client to connect to Amazon Polly to generate and download lines that you already know the user will require. The benefit is lower cost and faster response time. The disadvantage is that the client must store the files locally.

To import a package

  • Type the following lmbr_aws command:

    Copy
    lmbr_aws cloud-gem-tts import-tts-zip --download-path <absolute_path>

The download-path argument is the absolute path to the .zip file that you downloaded. Optionally, you can use the --import-as-wav argument to convert the .pcm files to .wav files so that you can import them into Wwise or other audio solutions.

The imported files are processed by the Lumberyard Asset Processor and added to your project for use in your game.