Tutorial: Get started with SageMaker Edge Manager - AWS IoT Greengrass
PrerequisitesSet up in SageMaker Edge ManagerCreate the sample componentsRun sample image classification inference

Tutorial: Get started with SageMaker Edge Manager

Important

SageMaker Edge Manager is being discontinued on April 26th, 2024. For more information about continuing to deploy your models to edge devices, see SageMaker Edge Manager end of life.

Amazon SageMaker Edge Manager is a software agent that runs on edge devices. SageMaker Edge Manager provides model management for edge devices so that you can package and use Amazon SageMaker Neo-compiled models directly on Greengrass core devices. By using SageMaker Edge Manager, you can also sample model input and output data from your core devices, and send that data to the AWS Cloud for monitoring and analysis. For more information about how SageMaker Edge Manager works on Greengrass core devices, see Use Amazon SageMaker Edge Manager on Greengrass core devices.

This tutorial shows you how to get started using SageMaker Edge Manager with AWS-provided sample components on an existing core device. These sample components use the SageMaker Edge Manager component as a dependency to deploy the Edge Manager agent, and perform inference using pre-trained models that were compiled using SageMaker Neo. For more information about the SageMaker Edge Manager agent, see SageMaker Edge Manager in the Amazon SageMaker Developer Guide.

To set up and use the SageMaker Edge Manager agent on an existing Greengrass core device, AWS provides example code that you can use to create the following sample inference and model components.

  • Image classification

    • com.greengrass.SageMakerEdgeManager.ImageClassification

    • com.greengrass.SageMakerEdgeManager.ImageClassification.Model

  • Object detection

    • com.greengrass.SageMakerEdgeManager.ObjectDetection

    • com.greengrass.SageMakerEdgeManager.ObjectDetection.Model

This tutorial shows you how to deploy the sample components and the SageMaker Edge Manager agent.

Prerequisites

To complete this tutorial, you must meet the following prerequisites:

  • A Greengrass core device running on Amazon Linux 2, a Debian-based Linux platform (x86_64 or Armv8), or Windows (x86_64). If you don't have one, see Tutorial: Getting started with AWS IoT Greengrass V2.

  • Python 3.6 or later, including pip for your version of Python, installed on your core device.

  • The OpenGL API GLX runtime (libgl1-mesa-glx) installed on your core device.

  • An AWS Identity and Access Management (IAM) user with administrator permissions.

  • An internet-enabled Windows, Mac, or Unix-like development computer that meets the following requirements:

  • The following S3 buckets created in the same AWS account and AWS Region as your Greengrass core device:

    • An S3 bucket to store the artifacts that are included in the sample inference and model components. This tutorial uses DOC-EXAMPLE-BUCKET1 to refer to this bucket.

    • An S3 bucket that you associate with your SageMaker edge device fleet. SageMaker Edge Manager requires an S3 bucket to create the edge device fleet, and to store sample data from running inference on your device. This tutorial uses DOC-EXAMPLE-BUCKET2 to refer to this bucket.

    For information about creating S3 buckets, see Getting started with Amazon S3.

  • The Greengrass device role configured with the following:

    • A trust relationship that allows credentials.iot.amazonaws.com and sagemaker.amazonaws.com to assume the role, as shown in the following IAM policy example.

      { 
  "Version": "2012-10-17",
  "Statement": [ 
    { 
      "Effect": "Allow", 
      "Principal": {
        "Service": "credentials.iot.amazonaws.com"
       }, 
      "Action": "sts:AssumeRole" 
    },
    { 
      "Effect": "Allow", 
      "Principal": {
        "Service": "sagemaker.amazonaws.com"
      }, 
      "Action": "sts:AssumeRole" 
    } 
  ] 
}

    • The AmazonSageMakerEdgeDeviceFleetPolicy IAM managed policy.

    • The AmazonSageMakerFullAccess IAM managed policy.

    • The s3:GetObject action for the S3 bucket that contains your component artifacts, as shown in the following IAM policy example.

      {
  "Version": "2012-10-17",
  "Statement": [
  {
    "Action": [
      "s3:GetObject"
    ],
    "Resource": [
      "arn:aws:s3:::DOC-EXAMPLE-BUCKET1/*"
    ],
    "Effect": "Allow"
  }
  ]
}

Set up your Greengrass core device in SageMaker Edge Manager

Edge device fleets in SageMaker Edge Manager are collections of logically grouped devices. To use SageMaker Edge Manager with AWS IoT Greengrass, you must create an edge device fleet that uses the same AWS IoT role alias as the Greengrass core device to which you deploy the SageMaker Edge Manager agent. Then, you must register the core device as part of that fleet.

Create an edge device fleet

To create an edge device fleet (console)

  1. In the Amazon SageMaker console, choose Edge Manager, and then choose Edge device fleets.

  2. On the Device fleets page, choose Create device fleet.

  3. Under Device fleet properties, do the following:

    • For Device fleet name, enter a name for your device fleet.

    • For IAM role, enter the Amazon Resource Name (ARN) of the AWS IoT role alias that you specified when setting up your Greengrass core device.

    • Disable the Create IAM role alias toggle.

  4. Choose Next.

  5. Under Output configuration, for S3 bucket URI, enter the URI of the S3 bucket that you want to associate with the device fleet.

  6. Choose Submit.

Register your Greengrass core device

To register your Greengrass core device as an edge device (console)

  1. In the Amazon SageMaker console, choose Edge Manager, and then choose Edge devices.

  2. On the Devices page, choose Register devices.

  3. Under Device properties, for Device fleet name, enter the name of the device fleet that you created, and then choose Next.

  4. Choose Next.

  5. Under Device source, for Device name, enter the AWS IoT thing name of your Greengrass core device.

  6. Choose Submit.

Create the sample components

To help you get started using the SageMaker Edge Manager component, AWS provides a Python script on GitHub that creates the sample inference and model components and uploads them to the AWS Cloud for you. Complete the following steps on a development computer.

To create the sample components

  1. Download the AWS IoT Greengrass component examples repository on GitHub to your development computer.

  2. Navigate to the downloaded /machine-learning/sagemaker-edge-manager folder.

    cd download-directory/machine-learning/sagemaker-edge-manager

  3. Run the following command to create and upload the sample components to the AWS Cloud.

    python3 create_components.py -r region -b DOC-EXAMPLE-BUCKET

    Replace region with the AWS Region where you created your Greengrass core device, and replace DOC-EXAMPLE-BUCKET1 with the name of the S3 bucket to store your component artifacts.

    Note

    By default, the script creates sample components for both image classification and object detection inference. To create components for only a specific type of inference, specify the -i ImageClassification | ObjectDetection argument.

Sample inference and model components for use with SageMaker Edge Manager are now created in your AWS account. To see the sample components in the AWS IoT Greengrass console, choose Components, and then under My components, search for the following components:

  • com.greengrass.SageMakerEdgeManager.ImageClassification

  • com.greengrass.SageMakerEdgeManager.ImageClassification.Model

  • com.greengrass.SageMakerEdgeManager.ObjectDetection

  • com.greengrass.SageMakerEdgeManager.ObjectDetection.Model

Run sample image classification inference

To run image classification inference using the AWS-provided sample components and the SageMaker Edge Manager agent, you must deploy these components to your core device. Deploying these components downloads a SageMaker Neo-compiled pre-trained Resnet-50 model and installs the SageMaker Edge Manager agent on your device. The SageMaker Edge Manager agent loads the model and publishes inference results on the gg/sageMakerEdgeManager/image-classification topic. To view these inference results, use the AWS IoT MQTT client in the AWS IoT console to subscribe to this topic.

Subscribe to the notifications topic

In this step, you configure the AWS IoT MQTT client in the AWS IoT console to watch MQTT messages published by the sample inference component. By default, the component publishes inference results on the gg/sageMakerEdgeManager/image-classification topic. Subscribe to this topic before you deploy the component to your Greengrass core device to see the inference results when the component runs for the first time.

To subscribe to the default notifications topic

  1. In the AWS IoT console navigation menu, choose Test, MQTT test client.

  2. Under Subscribe to a topic, in the Topic name box, enter gg/sageMakerEdgeManager/image-classification.

  3. Choose Subscribe.

Deploy the sample components

In this step, you configure and deploy the following components to your core device:

  • aws.greengrass.SageMakerEdgeManager

  • com.greengrass.SageMakerEdgeManager.ImageClassification

  • com.greengrass.SageMakerEdgeManager.ImageClassification.Model

  1. In the AWS IoT Greengrass console navigation menu, choose Deployments, and then choose the deployment for your target device that you want to revise.

  2. On the deployment page, choose Revise, and then choose Revise deployment.

  3. On the Specify target page, choose Next.

  4. On the Select components page, do the following:

    1. Under My components, select the following components:

      • com.greengrass.SageMakerEdgeManager.ImageClassification

      • com.greengrass.SageMakerEdgeManager.ImageClassification.Model

    2. Under Public components, turn off the Show only selected components toggle, and then select the aws.greengrass.SageMakerEdgeManager component.

    3. Choose Next.

  5. On the Configure components page, select the aws.greengrass.SageMakerEdgeManager component and do the following.

    1. Choose Configure component.

    2. Under Configuration update, in Configuration to merge, enter the following configuration.

      {
    "DeviceFleetName": "device-fleet-name",
    "BucketName": "DOC-EXAMPLE-BUCKET"
}

      Replace device-fleet-name with the name of the edge device fleet that you created, and replace DOC-EXAMPLE-BUCKET with the name of the S3 bucket that is associated with your device fleet.

    3. Choose Confirm, and then choose Next.

  6. On the Configure advanced settings page, keep the default configuration settings, and choose Next.

  7. On the Review page, choose Deploy

  1. On your development computer, create a deployment.json file to define the deployment configuration for your SageMaker Edge Manager components. This file should look like the following example.

    {
  "targetArn":"targetArn",
  "components": {
    "aws.greengrass.SageMakerEdgeManager": {
      "componentVersion": "1.0.x",
      "configurationUpdate": {
        "merge": "{\"DeviceFleetName\":\"device-fleet-name\",\"BucketName\":\"DOC-EXAMPLE-BUCKET2\"}"
      }
    },
    "com.greengrass.SageMakerEdgeManager.ImageClassification": {
      "componentVersion": "1.0.x",
      "configurationUpdate": {
      }
    }, 
    "com.greengrass.SageMakerEdgeManager.ImageClassification.Model": {
      "componentVersion": "1.0.x",
      "configurationUpdate": {
      }
    }, 
  }
}

    • In the targetArn field, replace targetArn with the Amazon Resource Name (ARN) of the thing or thing group to target for the deployment, in the following format:

      • Thing: arn:aws:iot:region:account-id:thing/thingName

      • Thing group: arn:aws:iot:region:account-id:thinggroup/thingGroupName

    • In the merge field, replace device-fleet-name with the name of the edge device fleet that you created. Then, replace DOC-EXAMPLE-BUCKET2 with the name of the S3 bucket that is associated with your device fleet.

    • Replace the component versions for each component with the latest available version.

  2. Run the following command to deploy the components on the device:

    aws greengrassv2 create-deployment \
    --cli-input-json file://path/to/deployment.json

The deployment can take several minutes to complete. In the next step, check the component log to verify that the deployment completed successfully and to view the inference results.

View inference results

After you deploy the components, you can view the inference results in the component log on your Greengrass core device and in the AWS IoT MQTT client in the AWS IoT console. To subscribe to the topic on which the component publishes inference results, see Subscribe to the notifications topic.

  • AWS IoT MQTT client—To view the results that the inference component publishes on the default notifications topic, complete the following steps:

    1. In the AWS IoT console navigation menu, choose Test, MQTT test client.

    2. Under Subscriptions, choose gg/sageMakerEdgeManager/image-classification.

     

  • Component log—To view the inference results in the component log, run the following command on your Greengrass core device. 

    sudo tail -f /greengrass/v2/logs/com.greengrass.SageMakerEdgeManager.ImageClassification.log

If you can't see inference results in the component log or in the MQTT client, the deployment failed or didn't reach the core device. This can occur if your core device isn't connected to the internet or doesn't have the right permissions to run the component. Run the following command on your core device to view the AWS IoT Greengrass Core software log file. This file includes logs from the Greengrass core device's deployment service.

sudo tail -f /greengrass/v2/logs/greengrass.log

For more information, see Troubleshooting machine learning inference.