GENCOST05-BP01 Create stopping conditions to control long-running workflows
Agentic workflows can be long-running, which incurs additional cost to your application. Develop controls to limit agents from running for extended periods of time without stopping.
Desired outcome: Maximum costs for an agent's runtime can be predicted based on the implemented stopping conditions.
Benefits of establishing this best practice: Measure overall efficiency - Agentic workflows can be long-running, which adds additional cost to your workload. By establishing stopping conditions for long-running agentic workflows, you can optimize resources, improve user experience, and optimize workload costs.
Level of risk exposed if this best practice is not established: High
Implementation guidance
For generative AI prompt flows where you lack control over the duration of the workflow, consider introducing a time-out mechanism or regaining control over the flow. This scenario is particularly common within agentic architectures. Agent architectures assist customers by taking on additional tasks. Sometimes these tasks can run for an extended duration, which may incur additional cost considerations, especially when they call external resources. Consider introducing a timeout over the agent to limit long-running processes from incurring costs unnecessarily. Additionally, evaluate asynchronous workflows orchestrated through events. Asynchronous workflows create opportunities to interrupt or halt long-running events after an extended duration. Consider the entire architecture before determining the best place to interrupt long-running workflows for cost savings.
Implementation steps
-
Determine the maximum time needed for an agent to complete its runtime.
-
Implement stopping conditions that enable an agent to run to the maximum duration.
-
Stopping conditions may be a timeout mechanism like the one in Amazon Bedrock.
-
Alternatively, stopping conditions may be implemented in the prompt flow layer or within a software abstraction layer.
-
Resources
Related practices:
Related guides, videos, and documentation:
Related examples:
