Create subtitles for your video-on-demand content to reach more audiences, globally - Content Localization on AWS

Create subtitles for your video-on-demand content to reach more audiences, globally

Publication date: November 2021

Content Localization on AWS helps extend the reach of your video-on-demand (VOD) content by efficiently creating accurate multi-language subtitles using Amazon Web Services (AWS) Artificial Intelligence (AI) services. You can make manual corrections to the automatically created subtitles and use advanced AWS AI service customization features to improve the results of the automation for your content. This solution is built on AWS Media Insights Engine (MIE), a framework that helps accelerate the development of serverless applications that process video, images, audio, and text with artificial intelligence services and multimedia services on AWS.

Localization is the process of taking video content that was created for audiences in one geography and transforming it to make it relevant and accessible to audiences in a new geography. Creating alternative language subtitle tracks is central to the localization process. With this solution, content owners can use a guided experience for automatically generating and correcting subtitles for videos in multiple languages using AWS AI services. Corrections made by solution users can be used as feedback to customize the results of AWS AI services for future workflows. This type of AI/ML (Machine Learning) workflow, which incorporates user corrections is referred to as human-in-the-loop.

Content localization workflows can make use of advanced customization features provided by Amazon Transcribe and Amazon Translate:

  • Amazon Transcribe custom vocabulary - you provide Amazon Transcribe with a list of terms that are specific to your content and how you want the terms to be displayed in transcripts.

  • Amazon Translate custom terminologies - you provide Amazon Translate with a list of terms or phrases in the source language content and specify how you want them to appear in the translated result.

  • Amazon Translate parallel data for active custom translation - you provide Amazon Translate with a list of parallel phrases: the source language and the phrase translated the way you want it. The parallel data customizes Amazon Translate models so they create more contextual translations based on the sample you provide.

Application users can manually correct the results of the automation at different points in the automated workflow and then invoke a new workflow to include their corrections in downstream processing. Corrections are tracked and can be used to update Amazon Transcribe custom vocabularies and Amazon Translate custom terminologies to improve future results.

Why use customizations and human-in-the-loop?

Automating the creation of translated subtitles using AI/ML helps to speed up the process of localization for your content, but there are still challenges to achieve the level of accuracy that is required for specific use cases. With natural language processing, many aspects of the content itself may determine the level of accuracy AI/ML analysis is capable of achieving. Some content characteristics that can impact transcription and translation accuracy include: domain specific language, speaker accents and dialects, new words recently introduced to common language, the need for contextual interpretation of ambiguous phases, and correct translation of proper names. AWS AI services provide a variety of features to help customize the results of the machine learning to specific content. Therefore, the workflow in this application seeks to provide users with a guided experience to use these customization features as an extension of their normal editing workflow.

This implementation guide discusses architectural considerations and configuration steps for deploying the Content Localization in AWS solution in the AWS Cloud. It includes links to an AWS CloudFormation template that launches and configures the AWS services required to deploy this solution using AWS best practices for security and availability.

The guide is intended for IT infrastructure architects and developers who have practical experience architecting in the AWS Cloud.