Capability 2. Providing secure access, usage, and implementation to generative AI RAG techniques

The following diagram illustrates the AWS services recommended for the Generative AI account for retrieval augmented generation (RAG) capability. The scope of this scenario is to secure RAG functionality.

The Generative AI account includes services that are required for storing embeddings in a vector database, storing conversations for users, and maintaining a prompt store along with a suite of required security services to implement security guardrails and centralized security governance. You should create Amazon S3 gateway endpoints for the model invocation logs, prompt store, and knowledge base data source buckets in Amazon S3 that the VPC environment is configured to access. You should also create a CloudWatch Logs gateway endpoint for the CloudWatch logs that the VPC environment is configured to access.

Rationale

Retrieval Augmented Generation (RAG) is a generative AI technique used where a system enhances its responses by retrieving information from an external, authoritative knowledge base before generating an answer. This process helps overcome the limitations of FMs by giving them access to up-to-date and context-specific data, which improves the accuracy and relevance of the generated responses. This use case refers to Scope 3 of the Generative AI Security Scoping Matrix. In Scope 3, your organization builds a generative AI application by using a pre-trained FM such as those offered in Amazon Bedrock. In this scope, you control your application and any customer data used by your application, whereas the FM provider controls the pre-trained model and its training data. 

When you give users access to Amazon Bedrock knowledge bases, you should address these key security considerations: 

• Secure access to the model invocation, knowledge bases, conversation history, and prompt store 

• Encryption of conversations, prompt store, and knowledge bases

• Alerts for potential security risks such as prompt injection or sensitive information disclosure

The next section discusses these security considerations and generative AI functionality.  

Design considerations

We recommend that you avoid customizing an FM with sensitive data (see the section on generative AI model customization later in this guide). Instead, use the RAG technique to interact with sensitive information. This method offers several advantages: 

• Tighter control and visibility. By keeping sensitive data separate from the model, you can exercise greater control and visibility over the sensitive information. The data can be easily edited, updated, or removed as needed, which helps ensure better data governance. 

• Mitigating sensitive information disclosure. RAG allows for more controlled interactions with sensitive data during model invocation. This helps reduce the risk of unintended disclosure of sensitive information, which could occur if the data were directly incorporated into the model's parameters. 

• Flexibility and adaptability. Separating sensitive data from the model provides greater flexibility and adaptability. As data requirements or regulations change, the sensitive information can be updated or modified without the need to retrain or rebuild the entire language model.

Amazon Bedrock knowledge bases

You can use Amazon Bedrock knowledge bases to build RAG applications by connecting FMs with your own data sources securely and efficiently. This feature uses Amazon OpenSearch Serverless as a vector store to retrieve relevant information from your data efficiently. The data is then used by the FM to generate responses. Your data is synchronized from Amazon S3 to the knowledge base, and embeddings are generated for efficient retrieval.

Security considerations

Generative AI RAG workloads face unique risks, including data exfiltration of RAG data sources and poisoning of RAG data sources with prompt injections or malware by threat actors. Amazon Bedrock knowledge bases offer robust security controls for data protection, access control, network security, logging and monitoring, and input/output validation that can help mitigate these risks.  

Remediations

Data protection

Encrypt your knowledge base data at rest by using an AWS Key Management Service (AWS KMS) customer managed key that you create, own, and manage. When you configure a data ingestion job for your knowledge base, encrypt the job with a customer managed key. If you opt to let Amazon Bedrock create a vector store in Amazon OpenSearch Service for your knowledge base, Amazon Bedrock can pass an AWS KMS key of your choice to Amazon OpenSearch Service for encryption.

You can encrypt sessions in which you generate responses from querying a knowledge base with an AWS KMS key. You store the data sources for your knowledge base in your S3 bucket. If you encrypt your data sources in Amazon S3 with a customer managed key, attach a policy to your Knowledge base service role. If the vector store that contains your knowledge base is configured with an AWS Secrets Manager secret, encrypt the secret with a customer managed key.

For more information and the policies to use, see Encryption of knowledge base resources in the Amazon Bedrock documentation.

Identity and access management

Create a custom service role for knowledge bases for Amazon Bedrock by following the principle of least privilege. Create a trust relationship that allows Amazon Bedrock to assume this role, and create and manage knowledge bases. Attach the following identity policies to the custom Knowledge base service role: 

• Permissions to access Amazon Bedrock models

• Permissions to access your data sources in Amazon S3

• Permissions to access your vector database in OpenSearch Service

• Permissions to access your Amazon Aurora database cluster (optional)

• Permissions to access a vector database that's configured with an AWS Secrets Manager secret (optional)

• Permissions for AWS to manage an AWS KMS key for transient data storage during data ingestion

• Permissions to chat with your document

• Permissions for AWS to manage a data source from another user's AWS account (optional). 

Knowledge bases support security configurations to set up data access policies for your knowledge base and network access policies for your private Amazon OpenSearch Serverless knowledge base. For more information, see Create a knowledge base and Service roles in the Amazon Bedrock documentation.

Input and output validation

Input validation is crucial for Amazon Bedrock knowledge bases. Use malware protection in Amazon S3 to scan files for malicious content before uploading them to a data source. For more information, see the AWS blog post Integrating Malware Scanning into Your Data Ingestion Pipeline with Antivirus for Amazon S3.

Identify and filter out potential prompt injections in user uploads to knowledge base data sources. Additionally, detect and redact personally identifiable information (PII) as another input validation control in your data ingestion pipeline. Amazon Comprehend can help detect and redact PII data in user uploads to knowledge base data sources. For more information, see Detecting PII entities in the Amazon Comprehend documentation.

We also recommend that you use Amazon Macie to detect and generate alerts on potential sensitive data in the knowledge base data sources, to enhance overall security and compliance. Implement Guardrails for Amazon Bedrock to help enforce content policies, block unsafe inputs/outputs, and help control model behavior based on your requirements.

Recommended AWS services

Amazon OpenSearch Serverless

Amazon OpenSearch Serverless is an on-demand, auto-scaling configuration for Amazon OpenSearch Service. An OpenSearch Serverless collection is an OpenSearch cluster that scales compute capacity based on your application's needs. Amazon Bedrock knowledge bases use Amazon OpenSearch Serverless for embeddings and Amazon S3 for the data sources that sync with the OpenSearch Serverless vector index. 

Implement strong authentication and authorization for your OpenSearch Serverless vector store. Implement the principle of least privilege, which grants only the necessary permissions to users and roles. 

With data access control in OpenSearch Serverless, you can allow users to access collections and indexes regardless of their access mechanisms or network sources. You manage access permissions through data access policies, which apply to collections and index resources. When you use this pattern, verify that the application propagates the identity of the user to the knowledge base, and the knowledge base enforces your role or attribute-based access controls. This is achieved by configuring the Knowledge Base service role with the principle of least privilege and controlling access to the role tightly. 

OpenSearch Serverless supports server-side encryption with AWS KMS to protect data at rest. Use a customer managed key to encrypt that data. To allow the creation of an AWS KMS key for transient data storage in the process of ingesting your data source, attach a policy to your knowledge bases for the Amazon Bedrock service role. 

Private access can apply to one or both of the following: OpenSearch Serverless-managed VPC endpoints and supported AWS services such as Amazon Bedrock. Use AWS PrivateLink to create a private connection between your VPC and OpenSearch Serverless endpoint services. Use network policy rules to specify Amazon Bedrock access.

Monitor OpenSearch Serverless by using Amazon CloudWatch, which collects raw data and processes it into readable, near real-time metrics.  OpenSearch Serverless is integrated with AWS CloudTrail, which captures  API calls for OpenSearch Serverless as events.  OpenSearch Service integrates with Amazon EventBridge to notify you of certain events that affect your domains. Third-party auditors can assess the security and compliance of OpenSearch Serverless as part of multiple AWS compliance programs.

Amazon S3

Store your data sources for your knowledge base in an S3 bucket. If you encrypted your data sources in Amazon S3 by using a custom AWS KMS key (recommended), attach  a policy to your Knowledge base service role.  Use malware protection in Amazon S3 to scan files for malicious content before uploading them to a data source. We also recommend that you host your model invocation logs and commonly used prompts as a prompt store in Amazon S3. All buckets should be encrypted with a customer managed key. For additional network security hardening, you can create a gateway endpoint for the S3 buckets that the VPC environment is configured to access. Access should be logged and monitored. Enable versioning if you have a business need to retain the history of Amazon S3 objects. Apply object-level immutability with Amazon S3 Object Lock. You can use resource-based policies to control access to your Amazon S3 files more tightly. 

Amazon Comprehend 

Amazon Comprehend uses natural language processing (NLP) to extract insights from the content of documents. You can use Amazon Comprehend to detect and redact PII entities in English or Spanish text documents. Integrate Amazon Comprehend into your data ingestion pipeline to automatically detect and redact PII entities from documents before you index them in your RAG knowledge base, to help ensure compliance and protect user privacy. Depending on the document types, you can use Amazon Textract to extract and send text to AWS Comprehend for analysis and redaction.

Amazon S3 enables you to encrypt your input documents when creating a text analysis, topic modeling, or custom Amazon Comprehend job. Amazon Comprehend integrates with AWS KMS to encrypt the data in the storage volume for Start* and Create* jobs, and it encrypts the output results of Start* jobs by using a customer managed key. We recommend that you use the aws:SourceArn and aws:SourceAccount global condition context keys in resource policies to limit the permissions that Amazon Comprehend gives another service to the resource. Use AWS PrivateLink to create a private connection between your VPC and Amazon Comprehend endpoint services. Implement identity-based policies for Amazon Comprehend with the principle of least privilege. Amazon Comprehend is integrated with AWS CloudTrail, which captures API calls for Amazon Comprehend as events. Third-party auditors can assess the security and compliance of Amazon Comprehend as part of multiple AWS compliance programs.

Amazon Macie 

Macie can help identify sensitive data in your knowledge bases that are stored as data sources, model invocation logs, and prompt store in S3 buckets. For Macie security best practices, see the Macie section earlier in this guidance. 

AWS KMS 

Use customer managed keys to encrypt the following: data ingestion jobs for your knowledge base, the Amazon OpenSearch Service vector database, sessions in which you generate responses from querying a knowledge base, model invocation logs in Amazon S3, and the S3 bucket that hosts the data sources. 

Use Amazon CloudWatch and Amazon CloudTrail as explained in the previous model inference section.