Using MXNet Elastic Inference accelerators on Amazon EC2 - Amazon Elastic Inference

Using MXNet Elastic Inference accelerators on Amazon EC2

When using Elastic Inference, you can use the same Amazon EC2 instance for models on two different frameworks. To do so, use the console to top the Amazon EC2 instance and restart it, instead of rebooting it. The Elastic Inference accelerator doesn't detach when you reboot the instance.

To use the Elastic Inference accelerator with MXNet

  1. Pull the MXNet-Elastic Inference image from Amazon Elastic Container Registry (Amazon ECR). To select an image, see Deep Learning Containers Images.

    docker pull 763104351884.dkr.ecr.<region>.amazonaws.com/mxnet-inference-eia:<image_tag>
  2. Run the container with the following command. You can get the <image_id> by running the docker images command.

    docker run -itd --name mxnet_inference -p 80:8080 -p 8081:8081 <image_id> \ mxnet-model-server --start --foreground \ --mms-config /home/model-server/config.properties \ --models resnet-152-eia=https://s3.amazonaws.com/model-server/model_archive_1.0/resnet-152-eia.mar
  3. Download the input image for the test.

    curl -O https://s3.amazonaws.com/model-server/inputs/kitten.jpg
  4. Begin inference using a query with the REST API.

    curl -X POST http://127.0.0.1:80/predictions/resnet-152-eia -T kitten.jpg
  5. The results should look something like the following.

    [ { "probability": 0.8582226634025574, "class": "n02124075 Egyptian cat" }, { "probability": 0.09160050004720688, "class": "n02123045 tabby, tabby cat" }, { "probability": 0.037487514317035675, "class": "n02123159 tiger cat" }, { "probability": 0.0061649843119084835, "class": "n02128385 leopard, Panthera pardus" }, { "probability": 0.003171598305925727, "class": "n02127052 lynx, catamount" } ]