OPS07-BP02 Identify and drive improvements
With clear drivers for improvement, you evaluate opportunities against business outcomes and data, rank them by their effect, and automate work where the return justifies the effort.
Desired outcome:
-
You collect operational data across your environment and correlate events and activities to business outcomes.
-
You evaluate improvement opportunities against defined drivers rather than reacting to the most recent event.
-
You compare metrics across environments and systems to find where improvement is needed, including partial failures that affect only some customers, requests, or locations.
-
You maintain a history of deployments and their outcomes to measure the effect of changes.
-
You measure the results of new features against the experience they were meant to deliver, judged from the customer's side rather than from internal health metrics alone.
Common anti-patterns:
-
Teams collect telemetry across the environment but never correlate events and activities, so the data drives no decisions.
-
Organizations gather detailed data at significant monitoring and logging cost yet never turn it into improvement actions.
-
Improvement priorities ignore business outcomes and reflect individual preference or recency bias.
-
Teams ship new features without measuring whether they achieved the intended result.
-
Improvement work starts only after an incident, so preventable issues accumulate between events.
-
Dashboards report the system as healthy while some customers experience failure, so a real degradation never becomes an improvement candidate.
Benefits of establishing this best practice:
-
You prioritize improvements that deliver the greatest benefit for the business.
-
You reduce the influence of emotion and recency by applying consistent criteria.
-
You respond to business drivers such as compliance deadlines and customer needs, not only technical faults.
-
You confirm whether a change actually improved anything by measuring its effect.
-
You direct effort toward the systems and processes where data shows the most opportunity.
-
You surface degradations that affect real customers but stay hidden from aggregate metrics, so the experience people actually get is what drives your backlog.
-
You keep a continuously refreshed supply of improvement candidates drawn from operational evidence, rather than waiting for the next incident to generate them.
Level of risk exposed if this best practice is not established: Medium
Implementation guidance
A driver for improvement is a documented reason to change a system, and only a supported desired outcome justifies the change. Most of these drivers fall into one of three categories:
-
Desired capabilities are new features or service options for something you cannot do today.
-
Unacceptable issues are bugs, vulnerabilities, and inefficiencies to remove, including rightsizing and optimization opportunities.
-
Compliance requirements are changes needed to meet regulation or policy, or to stay under support from a third party.
Apply consistent criteria to every candidate. Rank opportunities by their effect on cost, risk, customer experience, or revenue so priority does not depend on recency or individual preference.
Connect events and activities to business outcomes. Link each activity to its outcome, such as which deployment changed which metric and whether the metric moved the intended business result. Use these correlations to compare environments and systems and locate opportunities. Uncorrelated telemetry wastes monitoring and logging spend.
Aggregate metrics can hide a subset of users still affected. A change that improves a server-side average while the customer experience stays slow has not delivered the improvement. Judge each improvement by the customer experience, and treat the customer-side signal as authoritative when it disagrees with an internal one.
Some failures resist detection. A gray
failure is one with differential observability: your
health checks report the system as fine while customers observe it
as broken. Finding them takes metrics dimensioned the way failures
actually occur — per Availability Zone, per instance, per API, per
client. An impairment confined to one dimension looks like an
outlier. A related pattern is a failure that persists after its
trigger is gone. A retry storm, or a cache that has lost its
effectiveness, generates enough extra work to sustain the overload
on its own. Recovery needs deliberate intervention such as
shedding load
Build a reusable analysis capability to turn data into decisions. Query stored logs and operational data, transform it, and present it in dashboards that make trends visible to the people who prioritize work. A history of deployments and their outcomes gives you the before-and-after evidence to judge whether a change helped.
Drivers also come from outside your own telemetry. New service capabilities, security bulletins, and compliance program updates all signal changes to evaluate, and account-level advisory findings surface cost and resilience opportunities. Watch these sources on a cadence to keep your improvement backlog current.
AWS DevOps Agent
Implementation steps
-
Define your drivers for improvement: Classify each opportunity as a desired capability, an unacceptable issue, or a compliance requirement - record the desired outcome that justifies it. Reject changes that do not map to a supported outcome.
-
Establish feedback loops and correlate data to outcomes: Collect operational data across your environment and connect events and activities to business results. A feedback loop is only as good as the telemetry feeding it, so build on the instrumentation established in OPS01-BP01 Develop application telemetry and OPS01-BP02 Develop user experience telemetry, the latter of which supplies the customer-side signal you need to judge whether an improvement landed. Export operational logs to durable storage so you can analyze them over time, for example with Export log data to Amazon S3.
-
Instrument for partial failures, not just large scale incidents: Dimension your metrics along the boundaries where impairments actually occur, such as Availability Zone, instance, API operation, and client, so a problem confined to one dimension surfaces rather than being absorbed into an average. Add a customer-side view of the experience alongside your server-side health metrics. Outlier detection over log data, for example with CloudWatch Contributor Insights
, will point to the hosts or zones behaving differently from the rest. Record each detection gap you find as an improvement candidate in its own right. -
Build an analysis and reporting capability: Query and transform your operational data with services such as Amazon Athena and AWS Glue, then present trends in AWS Quick Sight dashboards, the business intelligence capability within AWS Quick
. Use the Cloud Intelligence Dashboards for prebuilt cost and usage views. -
Generate improvement candidates from operational evidence: Have AWS DevOps Agent evaluate your incident history on a schedule and propose changes that would help prevent classes of incidents, categorized across observability, infrastructure, governance, and code optimization. Keep the ones worth pursuing so they form a tracked backlog, and discard the rest with an explanation so future evaluations improve. Use to inform prioritization.
-
Monitor external sources of drivers: Track new capabilities through What's New with AWS
, review AWS Security Bulletins for vulnerabilities, and check AWS Trusted Advisor for cost and resilience findings. Fold compliance changes from AWS Compliance Programs into your backlog. -
Prioritize using consistent criteria: Rank opportunities by their effect on business outcomes such as cost, risk, customer experience, and revenue. Apply the same criteria to every candidate so that priority does not depend on recency or individual preference.
-
Measure results and automate where it pays off: After you implement an improvement, compare the outcome against the deployment history to confirm the change helped, and check the customer-side signal rather than only the internal metric. Automate recurring improvement work where the frequency and cost of manual effort justify the investment.
Resources
Related best practices:
Related documents:
Related videos:
Related examples:
Related services: