Getting image orientation and bounding box coordinates
Applications that use Amazon Rekognition Image commonly need to display the images that are detected by Amazon Rekognition Image operations and the boxes around detected faces. To display an image correctly in your application, you need to know the image's orientation and possibly correct it. For some .jpg files, the image's orientation is contained in the image's Exchangeable image file format (Exif) metadata. For other .jpg files and all .png files, Amazon Rekognition Image operations return the estimated orientation.
To display a box around a face, you need the coordinates for the face's bounding box. If the box isn't oriented correctly, you might need to adjust those coordinates. Amazon Rekognition Image face detection operations return bounding box coordinates for each detected face.
The following Amazon Rekognition Image operations return information for correcting an image's orientation and bounding box coordinates:
-
IndexFaces (Face model associated the collection must be version 3 or earlier)
This example shows how to get the following information for your code:
-
The estimated orientation of an image (if there is no orientation information in Exif metadata)
-
The bounding box coordinates for the faces detected in an image
-
Translated bounding box coordinates for bounding boxes that are affected by estimated image orientation
Use the information in this example to ensure that your images are oriented correctly and that bounding boxes are displayed in the correct location in your application.
Because the code used to rotate and display images and bounding boxes depends on the language and environment that you use, we don't explain how to display images and bounding boxes in your code, or how to get orientation information from Exif metadata.
Finding an image's orientation
To display an image correctly in your application, you might need to rotate it. The following image is oriented to 0 degrees and is displayed correctly.

However, the following image is rotated 90 degrees counterclockwise. To display it correctly, you need to find the orientation of the image and use that information in your code to rotate the image to 0 degrees.

Some images in .jpg format contain orientation information in Exif metadata. If
the value of the OrientationCorrection
field is null
in
the operation's response, the Exif metadata for the image contains the orientation.
In the Exif metadata, you can find the image's orientation in the
orientation
field. Although Amazon Rekognition Image identifies the presence of
image orientation information in Exif metadata, it does not provide access to it.
To
access the Exif metadata in an image, use a third-party library or write your own
code. For more information, see Exif
Version 2.32
Images in .png format do not have Exif metadata. For .jpg images that don't have
Exif metadata and for all .png images, Amazon Rekognition Image operations return
an estimated
orientation for the image in the OrientationCorrection
field. Estimated
orientation is measured counterclockwise and in increments of 90 degrees. For
example, Amazon Rekognition Image returns ROTATE_0 for an image that is oriented to
0 degrees and
ROTATE_90 for an image that is rotated 90 degrees counterclockwise.
When you know an image's orientation, you can write code to rotate and correctly display it.
Displaying bounding boxes
The Amazon Rekognition Image operations that analyze faces in an image also return the coordinates of the bounding boxes that surround the faces. For more information, see BoundingBox.
To display a bounding box around a face similar to the box shown in the following image in your application, use the bounding box coordinates in your code. The bounding box coordinates returned by an operation reflect the image's orientation. If you have to rotate the image to display it correctly, you might need to translate the bounding box coordinates.

Displaying bounding boxes when orientation information is not present in Exif metadata
If an image doesn't have Exif metadata, or if the orientation
field in the Exif metadata is not populated, Amazon Rekognition Image operations return
the
following:
-
An estimated orientation for the image
-
The bounding box coordinates oriented to the estimated orientation
If you need to rotate the image to display it correctly, you also need to rotate the bounding box.
For example, the following image is oriented at 90 degrees counterclockwise and shows a bounding box around the face. The bounding box is displayed using the coordinates for the estimated orientation returned from an Amazon Rekognition Image operation.

When you rotate the image to 0 degrees orientation, you also need to rotate the bounding box by translating the bounding box coordinates. For example, the following image has been rotated to 0 degrees from 90 degrees counterclockwise. The bounding box coordinates have not yet been translated, so the bounding box is displayed in the wrong position.

To rotate and display bounding boxes when orientation isn't present in Exif metadata
-
Call an Amazon Rekognition Image operation providing an input image with at least one face and with no Exif metadata orientation. For an example, see Detecting faces in an image.
-
Note the estimated orientation returned in the response's
OrientationCorrection
field. -
Rotate the image to 0 degrees orientation by using the estimated orientation you noted in step 2 in your code.
-
Translate the top and left bounding box coordinates to 0 degrees orientation and convert them to pixel points on the image in your code. Use the formula in the following list that matches the estimated orientation you noted in step 2.
Note the following definitions:
-
ROTATE_(n)
is the estimated image orientation returned by an Amazon Rekognition Image operation. -
<face>
represents information about the face that is returned by an Amazon Rekognition Image operation. For example, the ComparedFace data type that the RecognizeCelebrities operation returns contains bounding box information for faces detected in the source image. -
image.width
andimage.height
are pixel values for the width and height of the source image. -
The bounding box coordinates are a value between 0 and 1 relative to the image size. For example, for an image with 0-degree orientation, a
BoundingBox.left
value of 0.9 puts the left coordinate close to the right side of the image. To display the box, translate the bounding box coordinate values to pixel points on the image and rotate them to 0 degrees, as shown in each of the following formulas. For more information, see BoundingBox.
- ROTATE_0
-
left = image.width*BoundingBox.Left
top = image.height*BoundingBox.Top
- ROTATE_90
-
left = image.height * (1 - (<face>.BoundingBox.Top + <face>.BoundingBox.Height))
top = image.width * <face>.BoundingBox.Left
- ROTATE_180
-
left = image.width - (image.width*(<face>.BoundingBox.Left+<face>.BoundingBox.Width))
top = image.height * (1 - (<face>.BoundingBox.Top + <face>.BoundingBox.Height))
- ROTATE_270
-
left = image.height * BoundingBox.top
top = image.width * (1 - BoundingBox.Left - BoundingBox.Width)
-
-
Using the following formulas, calculate the bounding box's width and height as pixel ranges on the image in your code.
The width and height of a bounding box is returned in the
BoundingBox.Width
andBoundingBox.Height
fields. The width and height values range between 0 and 1 relative to the image size.image.width
andimage.height
are pixel values for the width and height of the source image.-
box width = image.width * (<face>.BoundingBox.Width)
box height = image.height * (<face>.BoundingBox.Height)
-
Display the bounding box on the rotated image by using the values calculated in steps 4 and 5.
Displaying bounding boxes when orientation information is present in Exif metadata
If an image's orientation is included in Exif metadata, Amazon Rekognition Image operations do the following:
-
Return null in the orientation correction field in the operation's response. To rotate the image, use the orientation provided in the Exif metadata in your code.
-
Return bounding box coordinates already oriented to 0 degrees. To show the bounding box in the correct position, use the coordinates that were returned. You do not need to translate them.
Example: Getting image orientation and bounding box coordinates for an image
The following example shows how to use the AWS SDK to get the estimated
orientation of an image and to translate bounding box coordinates for celebrities
detected
by the RecognizeCelebrities
operation.
The example loads an image from the local file system, calls the
RecognizeCelebrities
operation, determines the height and width of the
image, and calculates the bounding box coordinates of the face for the rotated
image. The example does not show how to process orientation information that is
stored in Exif metadata.
In the function main
, replace the value of photo
with the name and path of an
image that is stored locally in either .png or .jpg format.