Creating a custom vocabulary to improve speech recognition - Amazon Lex

Creating a custom vocabulary to improve speech recognition

You can give Amazon Lex V2 more information about how to process audio conversations with a bot by creating a custom vocabulary in a specific language. A custom vocabulary is a list of specific phrases that you want Amazon Lex V2 to recognize in the audio input. These are generally proper nouns or domain-specific words that Amazon Lex V2 doesn't recognize.

For example, suppose that you have a tech support bot. You can add "backup" to a custom vocabulary to help the bot transcribe the audio correctly as "backup," even when the audio sounds like "pack up." A custom vocabulary can also help recognize rare words in the audio such as "solvency" for financial services or proper nouns such as "Cognito" or "Monitron."

Custom vocabulary basics

  • A custom vocabulary works on the transcription of audio input to a bot. You must provide sample utterances to recognize an intent or slot value.

  • A custom vocabulary is unique to a specific language. You must configure custom vocabularies independently for each language. Custom vocabularies are supported only for the English (UK) and English (US) languages.

  • Custom vocabularies support only 8 kHz audio input. It is available with contact center integrations supported by Amazon Lex V2. The test window in the Amazon Lex V2 console does not support custom vocabularies as it uses 16 kHz audio input.

Amazon Lex V2 uses custom vocabularies to elicit both intents and slots. The same custom vocabulary file is used for intents and slots. You can selectively turn off the custom vocabulary capability for a slot.

Eliciting an intent – You can create a custom vocabulary for eliciting an intent. These phrases are used to transcription when your bot is determining the user's intent. For example, if you configured the phrase "backup" in your custom vocabulary, Amazon Lex V2 transcribes the user input to "can you please backup my photos?"—even when the audio sounds like "can you please pack up my photos." You can also specify the degree of boosting for each phrase by configuring a weight of 1, 2, or 3.

The custom vocabulary phrases used for improving transcription during intent elicitation don't affect transcriptions while eliciting slots. For more information about creating a custom vocabulary for eliciting intents, see Creating a custom vocabulary for eliciting intents.

Eliciting custom slots – You can use a custom vocabulary to improve slot recognition for audio conversations. To improve your Amazon Lex V2 bot's ability to recognize slot values, create a custom slot and add the slot values to the custom slot, then choose Use slot values as custom vocabulary. Examples of slot values include product names, catalogs, or proper nouns. You shouldn't use common words or phrases such as "yes" and "no" in custom vocabularies.

After the slot values are added, these values are used for improving slot recognition when the bot is expecting input for the custom slot. These values aren't used for transcription when eliciting an intent. For more information, see Adding slot types.

Best practices for creating a custom vocabulary

Eliciting an intent

  • Custom vocabularies work best when used to target specific words or phrases. Only add words to a custom vocabulary if they are not readily recognized by Amazon Lex V2.

  • Decide how much weight to give a word based on how often the word isn't recognized in the transcription and how rare the word is in the input. Difficult to pronounce words require a higher weight.

  • Use a representative test set to determine if a weight is appropriate. You can collect an audio test set by turning on audio logging in conversation logs.

  • Avoid using short words like "on," "it," "to," "yes," "no" in a custom vocabulary.

Eliciting a custom slot

  • Add the values to the custom slot type that you expect to be recognized. Add all the possible slot values for the custom slot type, no matter how common or rare the slot value is.

  • Enable the option only when the custom slot type contains a list of catalog values or entities such as product names or mutual funds.

  • Disable the option if the slot type is used to capture generic phrases such as "yes," "no," "I don't know," "maybe," or generic words such as "one," "two," "three."

  • Limit the number of slot values and synonyms to 500 or less for best performance.

Enter acronyms or other words whose letters should be pronounced individually as single letters separated by a period and a space. Don't use individual letters unless they are part of a phrase, such as "J. P. Morgan" or "A. W. S." You can use upper- or lower-case letters to define an acronym.

Creating a custom vocabulary for eliciting intents

You can use the Amazon Lex V2 console to create and manage a custom vocabulary, or you can use Amazon Lex V2 API operations.

To use the console to create a custom vocabulary

  1. Create a zip file with the phrases that you want to add to the custom vocabulary. For more information, see Creating a custom vocabulary file.

  2. Open the Amazon Lex V2 console at https://console.aws.amazon.com/lexv2/home

  3. From the list of bots, choose the one to which you want to add the custom vocabulary.

  4. On the bot detail page, from the Add languages section, choose View languages.

  5. From the list of languages, choose the language to which you want to add the custom vocabulary.

  6. In the Custom vocabulary section, choose Import.

  7. Browse to the file to import, enter a password if necessary, and then choose Import.

To use the API to create a custom vocabulary

  1. Create a zip file with the phrases that you want to add to the custom vocabulary. For more information, see Creating a custom vocabulary file.

  2. Use the CreateUploadUrl operation to get an import ID and upload URL. The response from the CreateUploadUrl operation looks like this:

    { "importId": "string", "uploadUrl": "string" }
  3. Use the StartImport operation to upload the custom vocabulary file to the upload URL. The import ID specifies a specific import.

  4. You can use the ListImports and DescribeImport operations to see the progress of importing your custom vocabulary.

Creating a custom vocabulary file

A custom vocabulary file is a tab-separated list of values that contain the phrase to recognize and a weight to give the boost. Phrases with a higher boost value are more likely to be used when they appear in the audio input.

The custom vocabulary file must be named CustomVocabulary.tsv, and must be compressed in a zip file before it can be imported. The zip file must be less than 300 MB in size. The maximum number of phrases in a custom vocabulary is 500.

  • phrase 1–4 words that should be recognized. Separate words in the phrase with spaces. You can't have duplicate phrases in the file. The phrase field is required.

  • weight – The degree to which the phrase recognition is boosted. The value is an integer 1, 2, or 3. If you don't specify a weight, the default value is 1. Decide on the weight based on how often the word isn't recognized in the transcription and on how rare the word is in the input.

The custom vocabulary file must contain a header row with the headers "phrase" and "weight". The headers can be in any order, but must use lower case letters.

The following is a custom vocabulary file. The required tab character to separate the phrase and the weight is represented by the text "[TAB]". If you use this example, replace the text with a tab character.

phrase[TAB]weight Newcastle[TAB]2 Hobart[TAB]2 U. Dub[TAB]1 W. S. U.[TAB]3 Issaquah Kennewick