Lumberyard
User Guide (Version 1.16)

Text-to-Speech Cloud Gem Portal

The Text-to-Speech Cloud Gem Portal (CGP) is a simple web interface that you can use to create text-to-speech (TTS) functionality for your game. This topic shows you how to perform the following tasks:

  • Add a character and specify a name, language, voice, and speech marks for it.

  • Add a speech line for the character.

  • Preview the audio and lip synchronization for the speech line, which is added to your speech library.

  • Add custom tags to your speech lines that you can use to filter searches of your speech library.

  • Create a .csv file with multiple speech lines.

  • Import the file into the Cloud Gem Portal for processing.

  • Download a .zip package of the voice and speech mark files generated by Amazon Polly.

  • Use the lmbr_aws command to import the generated voice packages into your project.

You can also configure whether game clients can request the generation of speech files and whether to cache them for more than a day. For more information, see Configuring Runtime Capabilities and Caching .

Prerequisites

This tutorial assumes that you have already performed the following steps to set up the Cloud Gem Portal (CGP) and the Text-to-Speech cloud gem:

If you don't meet the prerequisites, follow the steps in Enabling Gems to add Cloud Gem Text-to-Speech in the Project Configurator. For information on creating a project and deployment stacks, see Tutorial: Getting Started with Cloud Canvas.

Using the Text to Speech Cloud Gem Portal

In the Cloud Gem Portal, on the Cloud Gems page, choose Text to Speech.


        Choose Text to Speech in the Cloud Gem Portal.

The initial portal includes a Speech Library, Characters, and Generated Packages tabs. These are the only tabs that you need to get started. For information about the other tabs, see Configuring Runtime Capabilities and Caching and Text-to-Speech Troubleshooting Features.


        Text to Speech Cloud Gem initial screen

You can use the Text to Speech Cloud Gem Portal to create speech character content in a variety of ways. Use the workflow that suits you best. The following are some example workflows:

  • You create characters on the Characters tab with the voice and language characteristics that you want. Then use the Speech Library tab to create speech lines for the characters.

  • You create the speech lines that you want on the Speech Library tab. Then use the Characters tab to modify the default character that the cloud gem created for you for the speech lines.

  • You use the Speech Library tab to import speech lines from a .csv file that you created. The file can specify characters that you have already created in the portal, or you can let the cloud gem portal specify a default character for your speech lines.

Adding a Character

A text-to-speech character is defined by a name that you give the character, the language that the character speaks, and the male or female voice that the character uses. It can also have speech mark files that contain lip sync, closed captioning, and Speech Synthesis Markup Language (SSML) tags.

To add a character

  1. On the Characters tab of the Cloud Gem Portal, choose New Character to add a character to the characters table.

    
                Choose New Character in the Cloud Gem Portal
  2. In the fields that appear, specify the following for the character.

    • Name – By default, the new character is named NewCharacter_<total number of rows in the characters table>. Type a name for the character that has meaning in the context of your game.

    • Language – Choose the language that the character uses. You can choose from more than 20 languages. For a complete list of Amazon Polly languages and their corresponding voices, see Available Voices in the Amazon Polly Developer Guide.

    • Voice – Choose the name of the Amazon Polly male or female voice that you want for your character.

    • Speech Marks – You can select the following options. These options are enabled by default.

      • SSML to generate speech synthesis markup language tags.

      • Select Viseme to generate lip syncing information for the speech lines that you create.

      • Select Sentence/Word to generate closed captioning information.

  3. Choose Save.

  4. Choose Preview to hear the voice that you selected say, "Hi. My name is <name>."

Note

The next time that you add a character, the new character entry appears first in the list of characters. The values for a new character default to the values that you provided for the previous character.

Editing a Character

After you create a character, you can edit it with the following steps.

To edit a character

  1. Click an editable field to update it, and then choose Save.

    
                Edit fields inline for an existing character in the Cloud Gem
                  Portal.
  2. To alter the character's voice, choose the gear icon near Preview.

    1. Use the Advanced Settings dialog box to adjust the Volume, Rate, Pitch, and Timbre of the voice.

    2. Use the Language Override option to override the Native Language setting and have the character speak with a rendered accent.

    
                Advanced settings for changing a character's voice in the Cloud Gem
                  Portal.

Now you are ready to add a speech line to give the character something to say.

Adding a Speech Line

You can create speech lines one at a time in the Cloud Gem Portal.

To add an individual speech line

  1. In the Cloud Gem Portal, choose the Speech Library tab.

  2. Choose New Speech Line.

    
                Adding a speech line for your character in the Cloud Gem Portal.
  3. Specify the following for the speech line.

    • Character – Choose the character that will speak the speech line. You can choose from the characters that you created on the Characters tab.

    • Speech Line – Replace the default NewSpeechLine_<total number of rows in table> with the text that you want to convert to speech.

    • Preview – Choose the Preview icon to hear an audio preview of your text. If you selected the Viseme option for the character that you are using, the facial rig performs lip synchronization as well.

    • Tags – You can use custom tags to categorize your speech lines. You can use these tags to filter the speech lines that you want to view in the Cloud Gem Portal. To add one or more tags, enter the tag name and then press Enter after each tag.

    Warning

    Do not store sensitive customer information in speech lines or character definitions.

  4. Choose Save.

Note

The next time that you add a speech line, the new line appears first in the speech library. The Character and Tag values for the new speech line default to the values that you provided for the speech line that you last created.

Filtering Speech Lines

After you create multiple speech lines, click the filter icon for Character or Tags to filter them.


            Character and tag filters for speech lines.

In the filter dialog box, select the characters or tags that you want to filter for, and then choose Apply.


            Filtering by character

            Filtering by tags

For the tag filter, you can use the Match option to filter for Any or All of the tags that you select.

Deleting Speech Lines

You can delete speech lines one at a time or in multiples.

To delete a single speech line

  • Click the trash icon for the speech line.


            Deleting a speech line in the Cloud Gem Portal.

To delete a multiple speech lines

  • Select the speech lines that you want to delete, and then choose Delete.


            Deleting multiple speech lines in the Cloud Gem Portal.
Importing Speech Lines from a .csv File

Instead of creating your speech lines one by one in the Cloud Gem Portal, you can prepare a .csv file that contains all of the lines that you require. You can then use the Cloud Gem Portal to import the file.

Creating a Speech Line File

To create a speechline file in .csv format, see the following example. The example has rows that specify fields for character, speech line, time of day, mood, and stimulus.

Character,Speech Line,Time of Day,Mood,Stimulus Jane,Good Morning! How are you?,Morning,Happy, John,You stepped on my toe!,Afternoon,Angry,footSteppedOn Mary,Welcome to the Text to Speech Cloud Gem.,Night,Happy, Richard,Thank you for your help.,Afternoon,Happy,directionsReceived

The CGP treats the first line in the file as a header row. The second and following rows are data rows. Each data row should specify a character name and a line of speech for the character. A data row can have other fields that have meaning in the context of your game. You can map these fields to custom tags when you import the file.

Importing a Speech Line File

See the following steps to import a speech line file.

To import a .csv speech line file

  1. On the Speech Library tab, choose Import .csv., and then choose the .csv file to import.

  2. In the Import window, choose your custom mappings, and then choose Continue.

    
                  Mapping fields in the speech line .csv file.
  3. Select the speech lines that you want to import, or select the check box at the top to import them all.

    
                  Choosing speech lines to import from a .csv
                    file.
  4. Choose Continue. The imported lines appear in the speech library, where you can edit them as needed.

    
                  Imported speech lines

    Note

    If the character for a speech line does not exist, the cloud gem creates one based on the values of the last character that you created.

Downloading Speech Files

After you use the Cloud Gem Portal to create your speech line/character combinations in the AWS cloud, you can download a .zip file package of the files that were generated. You can then import the voice package into your project assets.

To download speech files

  1. In the Cloud Gem Portal, choose the Speech Library tab, select the speech lines that you want to download, and then choose Download Speech Files (.zip).

    
                Downloading speech files in the Cloud Gem Portal.
  2. Enter a name for the .zip file, and then choose Download.

    
                File name for download

    To ensure uniqueness, a UUID is added to the file name that you specify. The final file name format is <file_name><UUID>.zip.

  3. If your speech package is large, it can take some time to download. To check the status of a pending download, choose the cloud icon next to Download Speech Files (.zip).

    
                View pending speech file downloads in the Cloud Gem Portal.
  4. You can open the .zip file to view its contents. The file contains the voice (.pcm) files, the speech mark (.json) files, and a character_mapping.json file. It also contains a speech_line_definitions.csv file that contains the definitions of the speech lines that you selected.

    
                Viewing downloaded contents of the speech file.

    Several speech lines can share a voice file if they use the same voice and speech line. They can also share a speech marks file if they are using the same voice, speech line, and speech marks.

Downloading Generated Packages

When you download a speech file, a copy is saved on the Generated Packages tab. This creates a repository of generated packages that you can download again if needed.


            Downloading generated packages

To download a generated package

  • Select the packages that you want, and then choose Download Generated Packages.

To delete a generated package

  • Choose the trash icon for the package.

Importing the Generated Voice Packages Into Your Project

After you download the voice packages that you generated, you can import them into your project. This removes the need for the client to connect to Amazon Polly to generate and download lines that you already know the player will require. The benefit is lower cost and faster response time. The disadvantage is that the client must store the files locally.

To import a voice package

  • In the \dev directory, enter the following lmbr_aws command :

    lmbr_aws cloud-gem-tts import-tts-zip --download-path absolute_path

The download-path argument is the absolute path to the .zip file that you downloaded. Optionally, you can use the --import-as-wav argument to convert the .pcm files to .wav files so that you can import them into Wwise or other audio solutions.

The Lumberyard Asset Processor processes the imported files and adds them to your game project.