Create and Publish an AWS DeepLens Inference Lambda Function
This topic explains how to add an AWS Lambda inference function to your custom AWS DeepLens
project. Inference is the step where the model is shown images it has never seen before and
asked to make a prediction. The inference function optimizes the model to run on AWS DeepLens and
feeds each camera frame into the model to get predictions. For the inference function, you
use AWS Lambda
AWS DeepLens supports Lambda inference funtions written in Python 2.7 and Python 3.7. Python 3.7 requires requires device software v1.4.5. You will write the Lambda function in the browser and then deploy it to run on AWS DeepLens.
To create and publish an inference Lambda function for your AWS DeepLens project
-
Download the AWS DeepLens inference function template to your computer. Do not unzip the downloaded file.
The zip file contains the following:
-
lambda_function.py
— this file contains the Lambda function. AWS DeepLens will run this file after your AWS DeepLens project has been deployed. -
local_display.py
— this file contains a helper class to write the modified frames from the camera into the project video stream. -
greengrasssdk
— this folder contains the AWS IoT Greengrass SDK. It is used to send prediction results back to the cloud or to another IoT device.
You will be modifying
lambda_function.py
to pull frames from the camera, perform inference, and send predictions back through AWS IoT Greengrass. -
Sign in to the AWS Management Console and open the AWS Lambda console at https://console.aws.amazon.com/lambda/
. -
Choose Create function, then choose Author from Scratch.
-
In the Basic information section:
-
Under Function name, type a name for your Lambda function, for example,
deeplens-object-detection
.The function name must start with
deeplens
. -
Under Runtime, choose Python 2.7 or Python 3.7.
-
Under Permissions, expand Choose or create an execution role, if it's not already expanded.
-
Under Execution role, choose Use an existing role.
-
From the Existing role dropdown list, choose service-role/AWSDeepLensLambdaRole, which was created when you registered your device.
-
-
Scroll to the bottom of the page and choose Create function.
-
In the Function code section, do the following:
-
Under Actions, choose Upload a .zip file.
-
Under Function package, choose Upload, and then select
deeplens_inference_function_template.zip
.
-
-
Choose Save (on the upper-right corner of the Lambda console) to load the basic Lambda function into the code editor.
-
In the Lambda code editor, open
lambda_function.py
:Preserve import dependency
The
cv2
library has a dependency on theawscam
module. If you modify thelambda_function.py
template, be sure that it continues to import theawscam
module before thecv2
library.import json import awscam import mo import cv2 import greengrasssdk import os from local_display import LocalDisplay def lambda_handler(event, context): """Empty entry point to the Lambda function invoked from the edge.""" return def infinite_infer_run(): """ Run the DeepLens inference loop frame by frame""" # Load the model here while True: # Get a frame from the video stream ret, frame = awscam.getLastFrame() # Do inference with the model here # Send results back to IoT or output to video stream infinite_infer_run()
Let's examine this code line by line.
-
The
json
module lets your Lambda function work with JSON data and send results back through AWS IoT. -
The
awscam
module allows your Lambda function to use the AWS DeepLens device library to grab camera frames, perform inference with an optimized model and parse inference results. For more information, see Model Object. -
The
mo
module allows your Lambda function to access the AWS DeepLens model optimizer. It optimizes the model trained in the cloud to run efficiently on the DeepLens GPU. For more information, see Model Optimization (mo) Module. -
The
cv2
module lets your Lambda function access the Open CVlibrary used for image preprocessing. -
The
greengrasssdk
module exposes the AWS IoT Greengrass API for the Lambda function to send messages to the AWS Cloud, including sending operational status and inference results to AWS IoT. -
The
local_display
file contains a helper class to write to the project video stream. This helps you draw bounding boxes or text directly onto the DeepLens video stream. -
The
lambda_handler
is typically where you put the code for Lambda functions that are invoked once and then stopped. We want the Lambda function on AWS DeepLens to run and process frames continuously. We will leavelambda_handler
empty and define another functioninfinite_infer_run
that can run forever. -
The
infinite_infer_run
function contains awhile
loop that runs forever. It pulls usesawscam.getLastFrame
to pull frames from the camera on each iteration.
-
-
In this step, you add code to load the inference model and pass the camera frame to it to get predictions. For this example, we assume that you have trained a model to differentiate between a dog and a cat.
In the Lambda code editor, under the comment
# Load the model here
, add the following:# Model details input_width = 224 input_height = 224 model_name = 'image-classification' model_type = 'classification' output_map = {0: 'dog', 1: 'cat'} # Optimize the model error, model_path = mo.optimize(model_name,input_width,input_height) # Load the model onto the GPU. model = awscam.Model(model_path, {'GPU': 1})
AWS DeepLens uses the Intel OpenVino model optimizer to optimize the model trained in the cloud.
model_name
is the name of the model file you want to load. For models trained with Amazon SageMaker, the model typically has 2 files: a<model_name>-symbol.json
file and a<model_name>-###.params
file.input_width
andinput_height
refer to the size of the images used to train your network. During inference, the same-sized image is passed in.model_type
will tell theawscam
package how to parse the results. Other model types includessd
(single shot detector) andsegmentation
.output_map
helps turn the integer results from the model back to human-readable labels. -
In this step, you add code to pass camera frames through the model to get predictions.
In the Lambda code editor, under the comment
# Do inference with the model here
, add the following:frame_resize = cv2.resize(frame, (input_height, input_width)) predictions = model.doInference(frame_resize) parsed_inference_results = model.parseResult(model_type, predictions)
In this code, we resize the camera frame to be the same size as the image inputs used to trained the model. Next, we perform inference with the model on this resized frame. Finally, use the
awscam parseResult
to turn the predictions into a dictionary that can be easily parsed. For more information about the output ofparseResult
, see model.parseResult. -
In this step, you add code to send the results back to AWS IoT. This makes it easy to record the results in a database or trigger an action based on the predictions.
In the Lambda code editor, in the function
infinite_infer_run
, under""" Run the DeepLens inference loop frame by frame"""
, add the following:# Create an IoT client for sending to messages to the cloud. client = greengrasssdk.client('iot-data') iot_topic = '$aws/things/{}/infer'.format(os.environ['AWS_IOT_THING_NAME']) # Create a local display instance that will dump the image bytes to a FIFO # file that the image can be rendered locally. local_display = LocalDisplay('480p') local_display.start()
This code performs two tasks:
Instantiates an AWS IoT Greengrass SDK (
greengrasssdk
) to make the inference output available to the AWS Cloud. This includes sending process info and processed results to an AWS IoT topic (iot_topic
). The topic lets you view your AWS DeepLens output as JSON data instead of a video stream.Starts a thread (
local_display.start
) to feed parsed video frames for local display (LocalDisplay
). The display can be on device or in a web browser.
Add the following code after
# Send results back to IoT or output to video stream
to send the results back frame-by-frame:# Add the label of the top result to the frame used by local display. cv2.putText(frame, output_map[top_k[0]['label']], (10, 70), cv2.FONT_HERSHEY_SIMPLEX, 3, (255, 165, 20), 8) # Set the next frame in the local display stream. local_display.set_frame_data(frame) # Send the top k results to the IoT console via MQTT cloud_output = {} for obj in top_k: cloud_output[output_map[obj['label']]] = obj['prob'] client.publish(topic=iot_topic, payload=json.dumps(cloud_output))
-
Choose Save to save the updated Lambda function. Your function should look like the following:
import json import awscam import mo import cv2 import greengrasssdk import os from local_display import LocalDisplay def lambda_handler(event, context): """Empty entry point to the Lambda function invoked from the edge.""" return def infinite_infer_run(): """ Run the DeepLens inference loop frame by frame""" # Create an IoT client for sending to messages to the cloud. client = greengrasssdk.client('iot-data') iot_topic = '$aws/things/{}/infer'.format(os.environ['AWS_IOT_THING_NAME']) # Create a local display instance that will dump the image bytes to a FIFO # file that the image can be rendered locally. local_display = LocalDisplay('480p') local_display.start() # Load the model here # Model details input_width = 224 input_height = 224 model_name = 'image-classification' model_type = 'classification' output_map = {0: 'dog', 1: 'cat'} # Optimize the model error, model_path = mo.optimize(model_name,input_width,input_height) # Load the model onto the GPU. model = awscam.Model(model_path, {'GPU': 1}) while True: # Get a frame from the video stream ret, frame = awscam.getLastFrame() # Do inference with the model here # Resize frame to the same size as the training set. frame_resize = cv2.resize(frame, (input_height, input_width)) predictions = model.doInference(frame_resize) parsed_inference_results = model.parseResult(model_type, predictions) k = 2 # Get top k results with highest probabilities top_k = parsed_inference_results[model_type][0:k] print(top_k) # Send results back to IoT or output to video stream # Add the label of the top result to the frame used by local display. cv2.putText(frame, output_map[top_k[0]['label']], (10, 70), cv2.FONT_HERSHEY_SIMPLEX, 3, (255, 165, 20), 8) # Set the next frame in the local display stream. local_display.set_frame_data(frame) # Send the top k results to the IoT console via MQTT cloud_output = {} for obj in top_k: cloud_output[output_map[obj['label']]] = obj['prob'] client.publish(topic=iot_topic, payload=json.dumps(cloud_output)) infinite_infer_run()
-
From the Actions dropdown menu list, choose Publish new version. When you publish a function, it becomes available in the AWS DeepLens console and available to add to your custom project.