Core services: model access
Organizations must make a fundamental architectural decision regarding how agents access foundation models. This choice shapes security enforcement, operational governance, and the overall system architecture.
Two primary patterns exist for model access, each with advantages and disadvantages:
Cloud native
LLM gateway
Cloud native pattern
Direct access to cloud provider model services
Native tool integrations and action frameworks deeply tied to cloud ecosystems
Managed identity and access management specific to each platform
Orchestration and memory management services optimized for cloud-native architectures
Enterprise-grade observability, security and governance features embedded in managed services
LLM gateway pattern
Unified API interfaces across multiple model providers through a controlled intermediary
Uniform and centralized policy enforcement across model providers for security and compliance
Consistent monitoring, cost management, and API normalization
Decision factors
When choosing between these patterns, consider:
Governance maturity – organizations with strict compliance requirements or in highly regulated industries typically benefit from centralized control
Performance requirements – use cases requiring minimal latency may favor direct access
Multi-provider strategy – plans to use models from multiple providers benefit from the gateway's abstraction layer
Operational capacity – teams with limited resources may prefer cloud-native simplicity; those with established API management can leverage gateway benefits
Innovation velocity – direct access provides faster adoption of new model capabilities
Performance requirements – direct access avoids the additional routing layer latency introduced by gateways, though this overhead is often negligible compared to LLM inference time
Features of managed services - the rise of agentic AI and managed services (such as Amazon Bedrock Knowledge Bases) introduces platform-specific capabilities that are bringing value but are difficult to abstract
Organizations may also adopt a hybrid approach–implementing the gateway pattern for selected production applications while enabling cloud-native access for innovation workloads.
AWS implementation approaches
AWS supports both model access patterns through complementary services:
Cloud native pattern
Amazon Bedrock
– managed foundation model service providing unified API access to multiple models with built-in guardrails, security controls, and enterprise features Amazon SageMaker
– platform for deploying and hosting custom or third-party models with full control over infrastructure and model serving
Gateway pattern
AWS Guidance for Multi-Provider Generative AI Gateway
- reference architecture providing centralized access layer across Amazon Bedrock , Amazon SageMaker , and third-party model providers with unified usage tracking, cost management, rate limiting, model routing, and governance controls
Guardrails implementation
Regardless of pattern, organizations must implement guardrails that filter inappropriate content, enforce compliance requirements, validate inputs and outputs to prevent prompt injection, and support domain-specific customization.
Amazon BedrockGuardrails
