Using TensorFlow Models with Elastic Inference
Amazon Elastic Inference (Elastic Inference) is available only on instances that were launched with an Elastic Inference accelerator.
The Elastic Inference enabled version of TensorFlow allows you to use Elastic Inference accelerators with minimal changes to your TensorFlow code.
Topics
Elastic Inference Enabled TensorFlow
Preinstalled EI Enabled TensorFlow
The Elastic Inference enabled packages are available in the AWS Deep Learning AMIs. AWS Deep Learning AMIss come with supported TensorFlow version and
ei_for_tf
pre-installed. Elastic Inference enabled TensorFlow 2 requires
AWS Deep Learning AMIs v28 or higher. You also have Docker container options with Using Amazon Deep Learning Containers With Elastic Inference.
Installing EI Enabled TensorFlow
If you're not using a AWS Deep Learning AMIs instance, you can download the packages from
the Amazon S3 bucket
Install ei_for_tf
.
pip install -U ei_for_tf*.whl
If the TensorFlow version is lower than the required version, pip upgrades TensorFlow to the appropriate version. If the TensorFlow version is higher than the required version, there will be a warning about the incompatibility. Your program fails at run-time if the TensorFlow version incompatibility isn’t fixed.
Additional Requirements and Considerations
TensorFlow 2.0 Differences
Starting with TensorFlow 2.0, the Elastic Inference package is a separate pip wheel, instead of an enhanced TensorFlow pip wheel. The prefix for import statements for the Elastic Inference specific API have changed from tensorflow.contrib.ei to ei_for_tf.
To see the compatible TensorFlow version for a specific ei_for_tf version, see the
ei_for_tf_compatibility.txt
file in the Amazon
S3 bucket
Model Formats Supported
Elastic Inference supports the TensorFlow saved_model format via TensorFlow Serving.
Warmup
Elastic Inference TensorFlow Serving provides a warmup
Amazon Elastic Inference supports SageMaker Neo compiled TensorFlow models
Amazon Elastic Inference supports TensorFlow 2 models optimized by SageMaker Neo.
A pre-trained TensorFlow model can be compiled in SageMaker Neo with EIA as the
target device. The resulting model artifacts can be used for inference in
Elastic Inference Accelerators. This functionality only works for ei_for_tf
version
1.6 and greater. For more information, see
Use Elastic Inference with SageMaker Neo compiled models.