# Detecting video segments in stored video
<a name="segments"></a>

Amazon Rekognition Video provides an API that identifies useful segments of video, such as black frames and end credits. 

Viewers are watching more content than ever. In particular, Over-The-Top (OTT) and Video-On-Demand (VOD) platforms provide a rich selection of content choices anytime, anywhere, and on any screen. With proliferating content volumes, media companies are facing challenges in preparing and managing their content. This is crucial to providing a high-quality viewing experience and better monetizing content. Today, companies use large teams of trained human workforces to perform tasks such as the following.
+ Finding where the opening and end credits are in a piece of content
+ Choosing the right spots to insert advertisements, such as in silent black frame sequences
+ Breaking up videos into smaller clips for better indexing

These manual processes are expensive, slow, and can't scale to keep up with the volume of content that is produced, licensed, and retrieved from archives daily.

You can use Amazon Rekognition Video to automate operational media analysis tasks using fully managed, purpose-built video segment detection APIs powered by machine learning (ML). By using the Amazon Rekognition Video segment APIs, you can easily analyze large volumes of videos and detect markers such as black frames or shot changes. You get SMPTE (Society of Motion Picture and Television Engineers) timecodes, timestamps, and frame numbers from each detection. No ML experience is required. 

Amazon Rekognition Video analyzes videos stored in an Amazon Simple Storage Service (Amazon S3) bucket. The SMPTE timecodes that are returned are frame accurate – Amazon Rekognition Video provides the exact frame number of a detected segment of video, and handles various video frame rate formats automatically. You can use the frame accurate metadata from Amazon Rekognition Video to automate certain tasks completely, or to significantly reduce the review workload of trained human operators, so that they can focus on more creative work. You can perform tasks such as preparing content, inserting advertisements, and adding "binge-markers" to content at scale in the cloud. 

For information about pricing, see [Amazon Rekognition pricing](https://aws.amazon.com/rekognition/pricing/).

Amazon Rekognition Video segment detection supports two types of segmentation tasks — [Technical cues](#segment-technical-cue) detection and [Shot detection](#segment-shot-detection). 

**Topics**
+ [

## Technical cues
](#segment-technical-cue)
+ [

## Shot detection
](#segment-shot-detection)
+ [

## About the Amazon Rekognition Video Segment detection API
](#segment-api-intro)
+ [

# Using the Amazon Rekognition Segment API
](segment-api.md)
+ [

# Example: Detecting segments in a stored video
](segment-example.md)

## Technical cues
<a name="segment-technical-cue"></a>

A *technical cue* identifies black frames, color bars, opening credits, end credits, studio logos, and primary program content in a video. 

### Black frames
<a name="segment-black-frame"></a>

Videos often contain empty black frames with no audio that are used as cues to insert advertisements, or to mark the end of a program segment, such as a scene or opening credits. With Amazon Rekognition Video, you can detect black frame sequences to automate ad insertion, package content for VOD, and demarcate various program segments or scenes. Black frames with audio (such as fade outs or voiceovers) are considered as content and not returned. 

### Credits
<a name="segment-credits"></a>

Amazon Rekognition Video can automatically identify the exact frames where the opening and closing credits start and end for a movie or TV show. With this information, you can generate "binge markers" or interactive viewer prompts, such as "Next Episode’"or "Skip Intro," in video on demand (VOD) applications. You can also detect the first and last frame of program content in a video. Amazon Rekognition Video is trained to handle a wide variety of opening and end credit styles ranging from simple rolling credits to more challenging credits alongside content. 

### Color bars
<a name="segment-color-bar"></a>

Amazon Rekognition Video allows you to detect sections of video that display SMPTE color bars, which are a set of colors displayed in specific patterns to ensure color is calibrated correctly on broadcast monitors, programs, and on cameras. For more information about SMPTE color bars, see [SMPTE color bar](https://en.wikipedia.org/wiki/SMPTE_color_bars). This metadata is useful to prepare content for VOD applications by removing color bar segments from the content, or to detect issues such as loss of broadcast signals in a recording, when color bars are shown continuously as a default signal instead of content.

### Slates
<a name="segment-slates"></a>

Slates are sections of the video, typically near the beginning, that contain text metadata about the episode, studio, video format, audio channels, and more. Amazon Rekognition Video can identify the start and end of slates, making it easy to use the text metadata or remove the slate when preparing content for final viewing.

### Studio logos
<a name="segment-logos"></a>

Studio logos are sequences that show the logos or emblems of the production studio involved in making the show. Amazon Rekognition Video can detect these sequences so that users can review them to identify studios.

### Content
<a name="segment-content"></a>

Content is the portions of the TV show or movie that contain the program or related elements. Black frames, credits, color bars, slates, and studio logos are not considered content. Amazon Rekognition Video can detect the start and end of each content segment in the video, so you can find the program run time or specific segments.

Content segments include, but are not limited to, the following:
+ Program scenes between two ad breaks
+ A quick recap of the previous episode at the beginning of the video
+ Bonus post-credit content 
+ "Textless" content, such as a set of all program scenes that originally contained overlaid text, but where the text has been removed to support translation into other languages.

After Amazon Rekognition Video finishes detecting all of the content segments, you can apply domain knowledge or send them for human review to further categorize each segment. For example, if you use videos that always start with a recap, you could categorize the first content segment as a recap.

The following diagram shows technical cue segments on a show or movie's timeline. Note the color bars and opening credits, content segments such as the recap and main program, black frames throughout the video, and the end credits. 

![\[Color bars, recap segment, two program content segments, and black frames representing a show or movie timeline.\]](http://docs.aws.amazon.com/rekognition/latest/dg/images/technical-cue.png)


## Shot detection
<a name="segment-shot-detection"></a>

A shot is a series of interrelated consecutive pictures taken contiguously by a single camera and representing a continuous action in time and space. With Amazon Rekognition Video, you can detect the start, end, and duration of each shot, as well as a count for all the shots in a piece of content. You can use shot metadata for tasks such as the following. 
+ Creating promotional videos using selected shots.
+ Inserting advertisements in locations that don’t disrupt the viewer's experience, such as the middle of a shot when someone is speaking. 
+ Generating a set of preview thumbnails that avoid transitional content between shots.

A shot detection is marked at the exact frame where there is a hard cut to a different camera. If there is a soft transition from one camera to another, Amazon Rekognition Video omits the transition. This ensures that shot start and end times don’t include sections without actual content.

The following diagram illustrates shot detection segments on a strip of film. Note that each shot is identified by a cut from one camera angle or location to the next. 

![\[Seven numbered shots showing city streets, car dashboard, forest path, a child, a baby chick, sunset lake with photographer silhouette.\]](http://docs.aws.amazon.com/rekognition/latest/dg/images/shot-detection.png)


## About the Amazon Rekognition Video Segment detection API
<a name="segment-api-intro"></a>

To segment a stored video you use the asynchronous [StartSegmentDetection](https://docs.aws.amazon.com/rekognition/latest/APIReference/API_StartSegmentDetection.html) and [GetSegmentDetection](https://docs.aws.amazon.com/rekognition/latest/APIReference/API_GetSegmentDetection.html) API operations to start a segmentation job and fetch the results. Segment detection accepts videos stored in an Amazon S3 bucket and returns a JSON output. You can choose to detect only technical cues, only shot changes, or both together by configuring the `StartSegmentdetection` API request. You can also filter detected segments by setting thresholds for a minimum prediction confidence. For more information, see [Using the Amazon Rekognition Segment API](segment-api.md). For example code, see [Example: Detecting segments in a stored video](segment-example.md). 

# Using the Amazon Rekognition Segment API
<a name="segment-api"></a>

Amazon Rekognition Video segment detection in stored videos is an Amazon Rekognition Video asynchronous operation. The Amazon Rekognition Segment API is a composite API where you choose the type of analysis (technical cues or shot detection) from a single API call. For information about calling asynchronous operations, see [Calling Amazon Rekognition Video operations](api-video.md).

**Topics**
+ [

## Starting segment analysis
](#segment-api-start)
+ [

## Getting segment analysis results
](#segment-api-get)

## Starting segment analysis
<a name="segment-api-start"></a>

To start the detection of segments in a stored video call [StartSegmentDetection](https://docs.aws.amazon.com/rekognition/latest/APIReference/API_StartSegmentDetection.html). The input parameters are the same as other Amazon Rekognition Video operations with the addition of segment type selection and result filtering. For more information, see [Starting video analysis](api-video.md#api-video-start).

The following is example JSON passed by `StartSegmentDetection`. The request specifies that both technical cue and shot detection segments are detected. Different filters for the minimum detection confidence are requested for technical cue segments (90%) and shot detection segments (80%).

```
{
  "Video": {
    "S3Object": {
      "Bucket": "test_files",
      "Name": "test_file.mp4"
    }
    "SegmentTypes":["TECHNICAL_CUES", "SHOT"]
    "Filters": {
      "TechnicalCueFilter": {
         "MinSegmentConfidence": 90,
         "BlackFrame" : {
            "MaxPixelThreshold": 0.1,
            "MinCoveragePercentage": 95     
         }
      },
      "ShotFilter" : {
          "MinSegmentConfidence": 60
      }
  }
}
```

### Choosing a segment type
<a name="segment-feature-type"></a>

Use the `SegmentTypes` array input parameter to detect technical cue and/or shot detection segments in the input video. 
+ TECHNICAL\$1CUE — identifies frame-accurate timestamps for the start, end, and duration of technical cues (black frames, color bars, opening credits, end credits, studio logos, and primary program content) detected in a video. For example, you can use technical cues to find the start of the end credits. For more information, see [Technical cues](segments.md#segment-technical-cue).
+ SHOT — Identifies the start, end, and duration of a shot. For example, you can use shot detection to identify candidate shots for a final edit of a video. For more information, see [Shot detection](segments.md#segment-shot-detection).

### Filtering the analysis results
<a name="w2aac43c29b7c11"></a>

You can use the `Filters` ([StartSegmentDetectionFilters](https://docs.aws.amazon.com/rekognition/latest/APIReference/API_StartSegmentDetectionFilters.html)) input parameter to specify the minimum detection confidence returned in the response. Within `Filters`, use `ShotFilter` ([StartShotDetectionFilter](https://docs.aws.amazon.com/rekognition/latest/APIReference/API_StartShotDetectionFilter.html)) to filter detected shots. Use `TechnicalCueFilter` ([StartTechnicalCueDetectionFilter](https://docs.aws.amazon.com/rekognition/latest/APIReference/API_StartTechnicalCueDetectionFilter.html)) to filter technical cues. 

For example code, see [Example: Detecting segments in a stored video](segment-example.md).

## Getting segment analysis results
<a name="segment-api-get"></a>

Amazon Rekognition Video publishes the completion status of the video analysis to an Amazon Simple Notification Service topic. If the video analysis is successful, call [GetSegmentDetection](https://docs.aws.amazon.com/rekognition/latest/APIReference/API_GetSegmentDetection.html) to get results of the video analysis. 

The following is an example `GetSegmentDetection` request. The `JobId` is the job identifier returned from the call to `StartSegmentDetection`. For information about the other input parameters, see [Getting Amazon Rekognition Video analysis results](api-video.md#api-video-get). 

```
{
    "JobId": "270c1cc5e1d0ea2fbc59d97cb69a72a5495da75851976b14a1784ca90fc180e3",
    "MaxResults": 10,
    "NextToken": "XfXnZKiyMOGDhzBzYUhS5puM+g1IgezqFeYpv/H/+5noP/LmM57FitUAwSQ5D6G4AB/PNwolrw=="
}
```

`GetSegmentDetection` returns results for the requested analysis and general information about the stored video. 

### General information
<a name="segment-api-general"></a>

`GetSegmentDection` returns the following general information.
+ **Audio information** — The response includes audio metadata in an array, `AudioMetadata`, of [AudioMetadata](https://docs.aws.amazon.com/rekognition/latest/APIReference/API_AudioMetadata.html) objects. There can be multiple audio streams. Each `AudioMetadata` object contains metadata for a single audio stream. Audio information in an `AudioMetadata` objects includes the audio codec, the number of audio channels, the duration of the audio stream, and the sample rate. Audio metadata is returned in each page of information returned by `GetSegmentDetection`.
+ **Video information** – Currently, Amazon Rekognition Video returns a single [VideoMetadata](https://docs.aws.amazon.com/rekognition/latest/APIReference/API_VideoMetadata.html) object in the `VideoMetadata` array. The object contains information about the video stream in the input file that Amazon Rekognition Video chose to analyze. The `VideoMetadata` object includes the video codec, video format and other information. Video metadata is returned in each page of information returned by `GetSegmentDetection`.
+ **Paging information** – The example shows one page of segment information. You can specify how many elements to return in the `MaxResults` input parameter for `GetSegmentDetection`. If more results than `MaxResults` exist, `GetSegmentDetection` returns a token (`NextToken`) used to get the next page of results. For more information, see [Getting Amazon Rekognition Video analysis results](api-video.md#api-video-get).
+ **Request information** – The type of analysis requested in the call to `StartSegmentDetection` is returned in the `SelectedSegmentTypes` field.

### Segments
<a name="segment-api-technical-segments"></a>

Technical cues and shot information detected in a video is returned in an array, `Segments`, of [SegmentDetection](https://docs.aws.amazon.com/rekognition/latest/APIReference/API_SegmentDetection.html) objects. The array is sorted by the segment types (TECHNICAL\$1CUE or SHOT) specified in the `SegmentTypes` input parameter of `StartSegmentDetection`. Within each segment type the array is sorted by timestamp values. Each `SegmentDetection` object includes information about the type of detected segment (Technical cue or shot detection) and general information, such as the start time, end time, and the duration of the segment. 

Time information is returned in three formats.
+ 

**Milliseconds**  
The number of milliseconds since the start of the video. The fields `DurationMillis`, `StartTimestampMillis`, and `EndTimestampMillis` are in millisecond format.
+ 

**Timecode**  
Amazon Rekognition Video timecodes are in [SMPTE](https://en.wikipedia.org/wiki/SMPTE_timecode) format where each frame of video has a unique timecode value. The format is *hh:mm:ss:frame*. For example, a timecode value of 01:05:40:07, would be read as one hour, five minutes, forty seconds and seven frames. [Drop frame](https://en.wikipedia.org/wiki/SMPTE_timecode#Drop-frame_timecode) rate use cases are supported by Amazon Rekognition Video. The drop rate timecode format is *hh:mm:ss;frame*. The fields `DurationSMPTE`, `StartTimecodeSMPTE`, and `EndTimecodeSMPTE` are in timecode format.
+ 

**Frame Counters**  
The duration of each video segment is also expressed with the number of frames. The field `StartFrameNumber` gives the frame number at the start of a video segment, and `EndFrameNumber` gives the frame number at the end of a video segment. `DurationFrames` gives the total number of frames in a video segment. These values are calculated using a frame index that starts with 0.

You can use the `SegmentType` field to determine the type of a segment returned by Amazon Rekognition Video.
+ **Technical Cues** – the `TechnicalCueSegment` field is an [TechnicalCueSegment](https://docs.aws.amazon.com/rekognition/latest/APIReference/API_TechnicalCueSegment.html) object that contains the detection confidence and the type of a technical cue. The types of technical cue are `ColorBars`, `EndCredits`, `BlackFrames`, `OpeningCredits`, `StudioLogo`, `Slate`, and `Content`.
+ **Shot** – the `ShotSegment` field is a [ShotSegment](https://docs.aws.amazon.com/rekognition/latest/APIReference/API_ShotSegment.html) object than contains the detection confidence and an identifier for the shot segment within the video.

 The following example is the JSON response from `GetSegmentDetection`. 

```
{
    "SelectedSegmentTypes": [
        {
            "ModelVersion": "2.0",
            "Type": "SHOT"
        },
        {
            "ModelVersion": "2.0",
            "Type": "TECHNICAL_CUE"
        }
    ],
    "Segments": [
        {
            "DurationFrames": 299,
            "DurationSMPTE": "00:00:09;29",
            "StartFrameNumber": 0,
            "EndFrameNumber": 299,
            "EndTimecodeSMPTE": "00:00:09;29",
            "EndTimestampMillis": 9976,
            "StartTimestampMillis": 0,
            "DurationMillis": 9976,
            "StartTimecodeSMPTE": "00:00:00;00",
            "Type": "TECHNICAL_CUE",
            "TechnicalCueSegment": {
                "Confidence": 90.45006561279297,
                "Type": "BlackFrames"
            }
        },
        {
            "DurationFrames": 150,
            "DurationSMPTE": "00:00:05;00",
            "StartFrameNumber": 299,
            "EndFrameNumber": 449,
            "EndTimecodeSMPTE": "00:00:14;29",
            "EndTimestampMillis": 14981,
            "StartTimestampMillis": 9976,
            "DurationMillis": 5005,
            "StartTimecodeSMPTE": "00:00:09;29",
            "Type": "TECHNICAL_CUE",
            "TechnicalCueSegment": {
                "Confidence": 100.0,
                "Type": "Content"
            }
        },
        {
            "DurationFrames": 299,
            "ShotSegment": {
                "Index": 0,
                "Confidence": 99.9982681274414
            },
            "DurationSMPTE": "00:00:09;29",
            "StartFrameNumber": 0,
            "EndFrameNumber": 299,
            "EndTimecodeSMPTE": "00:00:09;29",
            "EndTimestampMillis": 9976,
            "StartTimestampMillis": 0,
            "DurationMillis": 9976,
            "StartTimecodeSMPTE": "00:00:00;00",
            "Type": "SHOT"
        },
        {
            "DurationFrames": 149,
            "ShotSegment": {
                "Index": 1,
                "Confidence": 99.9982681274414
            },
            "DurationSMPTE": "00:00:04;29",
            "StartFrameNumber": 300,
            "EndFrameNumber": 449,
            "EndTimecodeSMPTE": "00:00:14;29",
            "EndTimestampMillis": 14981,
            "StartTimestampMillis": 10010,
            "DurationMillis": 4971,
            "StartTimecodeSMPTE": "00:00:10;00",
            "Type": "SHOT"
        }
    ],
    "JobStatus": "SUCCEEDED",
    "VideoMetadata": [
        {
            "Format": "QuickTime / MOV",
            "FrameRate": 29.970029830932617,
            "Codec": "h264",
            "DurationMillis": 15015,
            "FrameHeight": 1080,
            "FrameWidth": 1920,
            "ColorRange": "LIMITED"

        }
    ],
    "AudioMetadata": [
        {
            "NumberOfChannels": 1,
            "SampleRate": 48000,
            "Codec": "aac",
            "DurationMillis": 15007
        }
    ]
}
```

For example code, see [Example: Detecting segments in a stored video](segment-example.md).

# Example: Detecting segments in a stored video
<a name="segment-example"></a>

The following procedure shows how to detect technical cue segments and shot detection segments in a video stored in an Amazon S3 bucket. The procedure also shows how to filter detected segments based on the confidence that Amazon Rekognition Video has in the accuracy of the detection.

The example expands on the code in [Analyzing a video stored in an Amazon S3 bucket with Java or Python (SDK)](video-analyzing-with-sqs.md) which uses an Amazon Simple Queue Service queue to get the completion status of a video analysis request. 

**To detect segments in a video stored in an Amazon S3 bucket (SDK)**

1. Perform [Analyzing a video stored in an Amazon S3 bucket with Java or Python (SDK)](video-analyzing-with-sqs.md).

1. Add the following to the code that you used in step 1.

------
#### [ Java ]

   1. Add the following imports.

      ```
      import com.amazonaws.services.rekognition.model.GetSegmentDetectionRequest;
      import com.amazonaws.services.rekognition.model.GetSegmentDetectionResult;
      import com.amazonaws.services.rekognition.model.SegmentDetection;
      import com.amazonaws.services.rekognition.model.SegmentType;
      import com.amazonaws.services.rekognition.model.SegmentTypeInfo;
      import com.amazonaws.services.rekognition.model.ShotSegment;
      import com.amazonaws.services.rekognition.model.StartSegmentDetectionFilters;
      import com.amazonaws.services.rekognition.model.StartSegmentDetectionRequest;
      import com.amazonaws.services.rekognition.model.StartSegmentDetectionResult;
      import com.amazonaws.services.rekognition.model.StartShotDetectionFilter;
      import com.amazonaws.services.rekognition.model.StartTechnicalCueDetectionFilter;
      import com.amazonaws.services.rekognition.model.TechnicalCueSegment;
      import com.amazonaws.services.rekognition.model.AudioMetadata;
      ```

   1. Add the following code to the class `VideoDetect`.

      ```
          //Copyright 2020 Amazon.com, Inc. or its affiliates. All Rights Reserved.
          //PDX-License-Identifier: MIT-0 (For details, see https://github.com/awsdocs/amazon-rekognition-developer-guide/blob/master/LICENSE-SAMPLECODE.)
      
      
          private static void StartSegmentDetection(String bucket, String video) throws Exception{
                  
              NotificationChannel channel= new NotificationChannel()
                      .withSNSTopicArn(snsTopicArn)
                      .withRoleArn(roleArn);
      
              float minTechnicalCueConfidence = 80F; 
              float minShotConfidence = 80F; 
                      
              StartSegmentDetectionRequest req = new StartSegmentDetectionRequest()
                      .withVideo(new Video()
                              .withS3Object(new S3Object()
                                      .withBucket(bucket)
                                      .withName(video)))
                      .withSegmentTypes("TECHNICAL_CUE" , "SHOT")
                      .withFilters(new StartSegmentDetectionFilters()
                              .withTechnicalCueFilter(new StartTechnicalCueDetectionFilter()
                                      .withMinSegmentConfidence(minTechnicalCueConfidence))
                              .withShotFilter(new StartShotDetectionFilter()
                                      .withMinSegmentConfidence(minShotConfidence)))
                      .withJobTag("DetectingVideoSegments")
                      .withNotificationChannel(channel);
      
              StartSegmentDetectionResult startLabelDetectionResult = rek.startSegmentDetection(req);
              startJobId=startLabelDetectionResult.getJobId();
              
          }
      
          private static void GetSegmentDetectionResults() throws Exception{
      
              int maxResults=10;
              String paginationToken=null;
              GetSegmentDetectionResult segmentDetectionResult=null;
              Boolean firstTime=true;
              
      
              do {
                  if (segmentDetectionResult !=null){
                      paginationToken = segmentDetectionResult.getNextToken();
                  }
      
                  GetSegmentDetectionRequest segmentDetectionRequest= new GetSegmentDetectionRequest()
                          .withJobId(startJobId)
                          .withMaxResults(maxResults)
                          .withNextToken(paginationToken);
      
                  segmentDetectionResult = rek.getSegmentDetection(segmentDetectionRequest);
                  
                  if(firstTime) {
                      System.out.println("\nStatus\n------");
                      System.out.println(segmentDetectionResult.getJobStatus());
                      System.out.println("\nRequested features\n------------------");
                       for (SegmentTypeInfo requestedFeatures : segmentDetectionResult.getSelectedSegmentTypes()) {
                          System.out.println(requestedFeatures.getType());
                      }
                       int count=1;
                       List<VideoMetadata> videoMetaDataList = segmentDetectionResult.getVideoMetadata();
                       System.out.println("\nVideo Streams\n-------------");
                       for (VideoMetadata videoMetaData: videoMetaDataList) {
                           System.out.println("Stream: " + count++);
                           System.out.println("\tFormat: " + videoMetaData.getFormat());
                           System.out.println("\tCodec: " + videoMetaData.getCodec());
                           System.out.println("\tDuration: " + videoMetaData.getDurationMillis());
                           System.out.println("\tFrameRate: " + videoMetaData.getFrameRate());
                       } 
      
                       
                       List<AudioMetadata> audioMetaDataList = segmentDetectionResult.getAudioMetadata();
                       System.out.println("\nAudio streams\n-------------");
      
                       count=1;
                       for (AudioMetadata audioMetaData: audioMetaDataList) {
                           System.out.println("Stream: " + count++);
                           System.out.println("\tSample Rate: " + audioMetaData.getSampleRate());
                           System.out.println("\tCodec: " + audioMetaData.getCodec());
                           System.out.println("\tDuration: " + audioMetaData.getDurationMillis());
                           System.out.println("\tNumber of Channels: " + audioMetaData.getNumberOfChannels());
                       }
                       System.out.println("\nSegments\n--------");
      
                      firstTime=false;
                  }
      
      
                  //Show segment information
      
                  List<SegmentDetection> detectedSegments= segmentDetectionResult.getSegments();
                  
                  for (SegmentDetection detectedSegment: detectedSegments) { 
                      
                     if (detectedSegment.getType().contains(SegmentType.TECHNICAL_CUE.toString())) {
                          System.out.println("Technical Cue");
                          TechnicalCueSegment segmentCue=detectedSegment.getTechnicalCueSegment();
                          System.out.println("\tType: " + segmentCue.getType()); 
                          System.out.println("\tConfidence: " + segmentCue.getConfidence().toString());
                      }
                     if (detectedSegment.getType().contains(SegmentType.SHOT.toString())) { 
                          System.out.println("Shot");
                          ShotSegment segmentShot=detectedSegment.getShotSegment();
                          System.out.println("\tIndex " + segmentShot.getIndex()); 
                          System.out.println("\tConfidence: " + segmentShot.getConfidence().toString());
                      }
                      long seconds=detectedSegment.getDurationMillis();
                      System.out.println("\tDuration : " + Long.toString(seconds) + " milliseconds");
                      System.out.println("\tStart time code: " + detectedSegment.getStartTimecodeSMPTE());
                      System.out.println("\tEnd time code: " + detectedSegment.getEndTimecodeSMPTE());
                      System.out.println("\tDuration time code: " + detectedSegment.getDurationSMPTE());
                      System.out.println();
                                      
                   } 
                         
              } while (segmentDetectionResult !=null && segmentDetectionResult.getNextToken() != null);
      
          }
      ```

   1. In the function `main`, replace the lines: 

      ```
              StartLabelDetection(amzn-s3-demo-bucket, video);
      
              if (GetSQSMessageSuccess()==true)
              	GetLabelDetectionResults();
      ```

      with:

      ```
              StartSegmentDetection(amzn-s3-demo-bucket, video);
      
              if (GetSQSMessageSuccess()==true)
              	GetSegmentDetectionResults();
      ```

------
#### [ Java V2 ]

   ```
   //snippet-start:[rekognition.java2.recognize_video_text.import]
   import software.amazon.awssdk.auth.credentials.ProfileCredentialsProvider;
   import software.amazon.awssdk.regions.Region;
   import software.amazon.awssdk.services.rekognition.RekognitionClient;
   import software.amazon.awssdk.services.rekognition.model.S3Object;
   import software.amazon.awssdk.services.rekognition.model.NotificationChannel;
   import software.amazon.awssdk.services.rekognition.model.Video;
   import software.amazon.awssdk.services.rekognition.model.StartTextDetectionRequest;
   import software.amazon.awssdk.services.rekognition.model.StartTextDetectionResponse;
   import software.amazon.awssdk.services.rekognition.model.RekognitionException;
   import software.amazon.awssdk.services.rekognition.model.GetTextDetectionResponse;
   import software.amazon.awssdk.services.rekognition.model.GetTextDetectionRequest;
   import software.amazon.awssdk.services.rekognition.model.VideoMetadata;
   import software.amazon.awssdk.services.rekognition.model.TextDetectionResult;
   import java.util.List;
   //snippet-end:[rekognition.java2.recognize_video_text.import]
   
   /**
   * Before running this Java V2 code example, set up your development environment, including your credentials.
   *
   * For more information, see the following documentation topic:
   *
   * https://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/get-started.html
   */
   public class DetectVideoSegments {
   
    private static String startJobId ="";
    public static void main(String[] args) {
   
        final String usage = "\n" +
            "Usage: " +
            "   <bucket> <video> <topicArn> <roleArn>\n\n" +
            "Where:\n" +
            "   bucket - The name of the bucket in which the video is located (for example, (for example, amzn-s3-demo-bucket). \n\n"+
            "   video - The name of video (for example, people.mp4). \n\n" +
            "   topicArn - The ARN of the Amazon Simple Notification Service (Amazon SNS) topic. \n\n" +
            "   roleArn - The ARN of the AWS Identity and Access Management (IAM) role to use. \n\n" ;
   
        if (args.length != 4) {
            System.out.println(usage);
            System.exit(1);
        }
   
        String bucket = args[0];
        String video = args[1];
        String topicArn = args[2];
        String roleArn = args[3];
   
        Region region = Region.US_WEST_2;
        RekognitionClient rekClient = RekognitionClient.builder()
            .region(region)
            .credentialsProvider(ProfileCredentialsProvider.create("profile-name"))
            .build();
   
        NotificationChannel channel = NotificationChannel.builder()
            .snsTopicArn(topicArn)
            .roleArn(roleArn)
            .build();
   
        startTextLabels(rekClient, channel, bucket, video);
        GetTextResults(rekClient);
        System.out.println("This example is done!");
        rekClient.close();
    }
   
    // snippet-start:[rekognition.java2.recognize_video_text.main]
    public static void startTextLabels(RekognitionClient rekClient,
                                   NotificationChannel channel,
                                   String bucket,
                                   String video) {
        try {
            S3Object s3Obj = S3Object.builder()
                .bucket(bucket)
                .name(video)
                .build();
   
            Video vidOb = Video.builder()
                .s3Object(s3Obj)
                .build();
   
            StartTextDetectionRequest labelDetectionRequest = StartTextDetectionRequest.builder()
                .jobTag("DetectingLabels")
                .notificationChannel(channel)
                .video(vidOb)
                .build();
   
            StartTextDetectionResponse labelDetectionResponse = rekClient.startTextDetection(labelDetectionRequest);
            startJobId = labelDetectionResponse.jobId();
   
        } catch (RekognitionException e) {
            System.out.println(e.getMessage());
            System.exit(1);
        }
    }
   
    public static void GetTextResults(RekognitionClient rekClient) {
   
        try {
            String paginationToken=null;
            GetTextDetectionResponse textDetectionResponse=null;
            boolean finished = false;
            String status;
            int yy=0 ;
   
            do{
                if (textDetectionResponse !=null)
                    paginationToken = textDetectionResponse.nextToken();
   
                GetTextDetectionRequest recognitionRequest = GetTextDetectionRequest.builder()
                    .jobId(startJobId)
                    .nextToken(paginationToken)
                    .maxResults(10)
                    .build();
   
                // Wait until the job succeeds.
                while (!finished) {
                    textDetectionResponse = rekClient.getTextDetection(recognitionRequest);
                    status = textDetectionResponse.jobStatusAsString();
   
                    if (status.compareTo("SUCCEEDED") == 0)
                        finished = true;
                    else {
                        System.out.println(yy + " status is: " + status);
                        Thread.sleep(1000);
                    }
                    yy++;
                }
   
                finished = false;
   
                // Proceed when the job is done - otherwise VideoMetadata is null.
                VideoMetadata videoMetaData=textDetectionResponse.videoMetadata();
                System.out.println("Format: " + videoMetaData.format());
                System.out.println("Codec: " + videoMetaData.codec());
                System.out.println("Duration: " + videoMetaData.durationMillis());
                System.out.println("FrameRate: " + videoMetaData.frameRate());
                System.out.println("Job");
   
                List<TextDetectionResult> labels= textDetectionResponse.textDetections();
                for (TextDetectionResult detectedText: labels) {
                    System.out.println("Confidence: " + detectedText.textDetection().confidence().toString());
                    System.out.println("Id : " + detectedText.textDetection().id());
                    System.out.println("Parent Id: " + detectedText.textDetection().parentId());
                    System.out.println("Type: " + detectedText.textDetection().type());
                    System.out.println("Text: " + detectedText.textDetection().detectedText());
                    System.out.println();
                }
   
            } while (textDetectionResponse !=null && textDetectionResponse.nextToken() != null);
   
        } catch(RekognitionException | InterruptedException e) {
            System.out.println(e.getMessage());
            System.exit(1);
        }
    }
    // snippet-end:[rekognition.java2.recognize_video_text.main]
   }
   ```

------
#### [ Python ]

   1. Add the following code to the class `VideoDetect` that you created in step 1.

      ```
      # Copyright 2020 Amazon.com, Inc. or its affiliates. All Rights Reserved.
      # PDX-License-Identifier: MIT-0 (For details, see https://github.com/awsdocs/amazon-rekognition-developer-guide/blob/master/LICENSE-SAMPLECODE.)
      
          def StartSegmentDetection(self):
      
              min_Technical_Cue_Confidence = 80.0
              min_Shot_Confidence = 80.0
              max_pixel_threshold = 0.1
              min_coverage_percentage = 60
      
              response = self.rek.start_segment_detection(
                  Video={"S3Object": {"Bucket": self.bucket, "Name": self.video}},
                  NotificationChannel={
                      "RoleArn": self.roleArn,
                      "SNSTopicArn": self.snsTopicArn,
                  },
                  SegmentTypes=["TECHNICAL_CUE", "SHOT"],
                  Filters={
                      "TechnicalCueFilter": {
                          "BlackFrame": {
                              "MaxPixelThreshold": max_pixel_threshold,
                              "MinCoveragePercentage": min_coverage_percentage,
                          },
                          "MinSegmentConfidence": min_Technical_Cue_Confidence,
                      },
                      "ShotFilter": {"MinSegmentConfidence": min_Shot_Confidence},
                  }
              )
      
              self.startJobId = response["JobId"]
              print(f"Start Job Id: {self.startJobId}")
      
          def GetSegmentDetectionResults(self):
              maxResults = 10
              paginationToken = ""
              finished = False
              firstTime = True
      
              while finished == False:
                  response = self.rek.get_segment_detection(
                      JobId=self.startJobId, MaxResults=maxResults, NextToken=paginationToken
                  )
      
                  if firstTime == True:
                      print(f"Status\n------\n{response['JobStatus']}")
                      print("\nRequested Types\n---------------")
                      for selectedSegmentType in response['SelectedSegmentTypes']:
                          print(f"\tType: {selectedSegmentType['Type']}")
                          print(f"\t\tModel Version: {selectedSegmentType['ModelVersion']}")
      
                      print()
                      print("\nAudio metadata\n--------------")
                      for audioMetadata in response['AudioMetadata']:
                          print(f"\tCodec: {audioMetadata['Codec']}")
                          print(f"\tDuration: {audioMetadata['DurationMillis']}")
                          print(f"\tNumber of Channels: {audioMetadata['NumberOfChannels']}")
                          print(f"\tSample rate: {audioMetadata['SampleRate']}")
                      print()
                      print("\nVideo metadata\n--------------")
                      for videoMetadata in response["VideoMetadata"]:
                          print(f"\tCodec: {videoMetadata['Codec']}")
                          print(f"\tColor Range: {videoMetadata['ColorRange']}")
                          print(f"\tDuration: {videoMetadata['DurationMillis']}")
                          print(f"\tFormat: {videoMetadata['Format']}")
                          print(f"\tFrame rate: {videoMetadata['FrameRate']}")
                          print("\nSegments\n--------")
      
                      firstTime = False
      
                  for segment in response['Segments']:
      
                      if segment["Type"] == "TECHNICAL_CUE":
                          print("Technical Cue")
                          print(f"\tConfidence: {segment['TechnicalCueSegment']['Confidence']}")
                          print(f"\tType: {segment['TechnicalCueSegment']['Type']}")
      
                      if segment["Type"] == "SHOT":
                          print("Shot")
                          print(f"\tConfidence: {segment['ShotSegment']['Confidence']}")
                          print(f"\tIndex: " + str(segment["ShotSegment"]["Index"]))
      
                      print(f"\tDuration (milliseconds): {segment['DurationMillis']}")
                      print(f"\tStart Timestamp (milliseconds): {segment['StartTimestampMillis']}")
                      print(f"\tEnd Timestamp (milliseconds): {segment['EndTimestampMillis']}")
                      
                      print(f"\tStart timecode: {segment['StartTimecodeSMPTE']}")
                      print(f"\tEnd timecode: {segment['EndTimecodeSMPTE']}")
                      print(f"\tDuration timecode: {segment['DurationSMPTE']}")
      
                      print(f"\tStart frame number {segment['StartFrameNumber']}")
                      print(f"\tEnd frame number: {segment['EndFrameNumber']}")
                      print(f"\tDuration frames: {segment['DurationFrames']}")
      
                      print()
      
                  if "NextToken" in response:
                      paginationToken = response["NextToken"]
                  else:
                      finished = True
      ```

   1. In the function `main`, replace the lines:

      ```
          analyzer.StartLabelDetection()
          if analyzer.GetSQSMessageSuccess()==True:
              analyzer.GetLabelDetectionResults()
      ```

      with:

      ```
          analyzer.StartSegmentDetection()
          if analyzer.GetSQSMessageSuccess()==True:
              analyzer.GetSegmentDetectionResults()
      ```

------
**Note**  
If you've already run a video example other than [Analyzing a video stored in an Amazon S3 bucket with Java or Python (SDK)](video-analyzing-with-sqs.md), the code to replace might be different.

1. Run the code. Information about the segments detected in the input video are displayed.