AWS services in this solution
AWS service | Description |
---|---|
AWS Batch |
Core. The solution uses AWS Batch processing for extracting Apache Parquet and PNG files. |
Amazon DynamoDB |
Core. This solution creates a DynamoDB table for tracking the batch of rosbag files, and another DynamoDB table for tracking the metadata that is generated after identifying lanes and objects from drive files. This solution uses the second DynamoDB table for downstream consuming and querying purposes. |
Amazon EMR Serverless | Core. The solution loads outputs from object detection and LaneDet into data frames and integration with the Parquet data by using Amazon EMR Serverless. This provides metadata for querying. |
Amazon
MWAA |
Core. Amazon MWAA orchestrates the workflow of uploading the rosbag files, extracting images and lanes from the files using open source YOLO model, and writing the required metadata to DynamoDB table. |
Amazon OpenSearch Service |
Core. This solution creates an OpenSearch Service cluster for advanced querying purposes. This cluster uses the metadata from DynamoDB. |
Amazon S3 |
Core. The solution stages all rosbag input files in a source S3 bucket for raw data, and the intermediate artifacts (such as Apache Parquet, PNG, and JSON files) in an intermediate S3 bucket for staged data. |
Amazon SageMaker |
Core. The solution runs object detection and LaneDet jobs by using SageMaker processing jobs. |
AWS CodeBuild |
Supporting. AWS CodeBuild manages module deployments for this solution. |
Amazon EC2 |
Supporting. A micro Amazon EC2 instance acts as a secure proxy to the OpenSearch Dashboard. |
Amazon ECR |
Supporting. The
solution builds and stores container images related to the
|
AWS IAM |
Supporting. This solution creates IAM roles for all the AWS services that require permissions to communicate with other AWS APIs. This solution uses least-privileged IAM policies. |
AWS Lambda |
Supporting. This solution uses a Lambda function to load the data from the DynamoDB table into an OpenSearch Service cluster. |
AWS Systems Manager |
Supporting. The solution allows Systems Manger tunneling for port forwarding to an Amazon EC2 instance in a private Amazon VPC subnet. This allows secure access to the OpenSearch Dashboard. |