Core services: model access

Organizations must make a fundamental architectural decision regarding how agents access foundation models. This choice shapes security enforcement, operational governance, and the overall system architecture.

Two primary patterns exist for model access, each with advantages and disadvantages:

• Cloud native

• LLM gateway

Cloud native pattern

• Direct access to cloud provider model services

• Native tool integrations and action frameworks deeply tied to cloud ecosystems

• Managed identity and access management specific to each platform

• Orchestration and memory management services optimized for cloud-native architectures

• Enterprise-grade observability, security and governance features embedded in managed services

LLM gateway pattern

• Unified API interfaces across multiple model providers through a controlled intermediary

• Uniform and centralized policy enforcement across model providers for security and compliance

• Consistent monitoring, cost management, and API normalization

Decision factors

When choosing between these patterns, consider:

• Governance maturity – organizations with strict compliance requirements or in highly regulated industries typically benefit from centralized control

• Performance requirements – use cases requiring minimal latency may favor direct access

• Multi-provider strategy – plans to use models from multiple providers benefit from the gateway's abstraction layer

• Operational capacity – teams with limited resources may prefer cloud-native simplicity; those with established API management can leverage gateway benefits

• Innovation velocity – direct access provides faster adoption of new model capabilities

• Performance requirements – direct access avoids the additional routing layer latency introduced by gateways, though this overhead is often negligible compared to LLM inference time

• Features of managed services - the rise of agentic AI and managed services (such as Amazon Bedrock Knowledge Bases) introduces platform-specific capabilities that are bringing value but are difficult to abstract

Organizations may also adopt a hybrid approach–implementing the gateway pattern for selected production applications while enabling cloud-native access for innovation workloads.

AWS implementation approaches

AWS supports both model access patterns through complementary services:

Cloud native pattern

• Amazon Bedrock – managed foundation model service providing unified API access to multiple models with built-in guardrails, security controls, and enterprise features

• Amazon SageMaker – platform for deploying and hosting custom or third-party models with full control over infrastructure and model serving

Gateway pattern

• AWS Guidance for Multi-Provider Generative AI Gateway - reference architecture providing centralized access layer across Amazon Bedrock, Amazon SageMaker, and third-party model providers with unified usage tracking, cost management, rate limiting, model routing, and governance controls

Guardrails implementation

Regardless of pattern, organizations must implement guardrails that filter inappropriate content, enforce compliance requirements, validate inputs and outputs to prevent prompt injection, and support domain-specific customization.

Amazon BedrockGuardrails provides managed capabilities for content filtering, denied topics, word filters, and sensitive information redaction.