6 – Design resilience for analytics workload
How do you design analytics workloads to withstand and mitigate failures?
|
ID |
Priority |
Best practice |
|---|---|---|
|
☐ BP 6.1 |
Required | Create an illustration of data flow dependencies. |
|
☐ BP 6.2 |
Required | Monitor analytics systems to detect analytics or extract, transform and load (ETL) job failures. |
|
☐ BP 6.3 |
Required | Notify stakeholders about analytics or ETL job failures. |
|
☐ BP 6.4 |
Recommended | Automate the recovery of analytics and ETL job failures. |
|
☐ BP 6.5 |
Recommended | Build a disaster recovery (DR) plan for the analytics infrastructure and the data. |
For more details, refer to the following documentation:
-
AWS Glue Developer Guide: Running and Monitoring AWS Glue
-
AWS Glue Developer Guide: Monitoring with Amazon CloudWatch
-
AWS Glue Developer Guide: Monitoring AWS Glue Using Amazon CloudWatch Metrics
-
AWS Prescriptive Guidance – Patterns: Orchestrate an ETL pipeline with validation, transformation, and partitioning using AWS Step Functions
-
AWS Support Knowledge Center: How can I use a Lambda function to receive SNS alerts
when an AWS Glue job fails a retry? -
AWS Glue Developer Guide: Repairing and Resuming a Workflow Run
-
AWS Data Pipeline Developer Guide: Cascading Failures and Reruns
