Use AWS Generative AI CDK constructs to speed up app development

Assemble and deploy the infrastructure for a RAG solution using AWS CDK for Python

In this blog, we will use the AWS Generative AI Constructs Library to deploy a complete RAG application composed of the following components:

• Knowledge Bases for Amazon Bedrock: This is the foundation for the RAG solution.
• OpenSearch Serverless collection: It supports the vector search collection type that provides similarity search capability.
• An S3 bucket: This will act as the data source for the Knowledge Base.
• AWS Lambda function (written in Python) along with an API Gateway that uses the RetrieveAndGenerate API to query the knowledge base and generate responses from the information it retrieves.

Introduction

AWS Cloud Development Kit (AWS CDK) is an open-source software development framework for defining cloud infrastructure in code and provisioning it through AWS CloudFormation. You can use any of these supported programming languages (TypeScript, JavaScript, Python, Java, C#/.Net, and Go) to define reusable cloud components known as constructs, that represent one or more AWS CloudFormation resources and their configuration.

Constructs from the AWS Construct Library are categorized into three levels with each level offers an increasing level of abstraction.

• L1 constructs are the lowest-level construct and offer no abstraction. Each L1 construct maps directly to a single AWS CloudFormation resource.
• L2 constructs provide a higher-level abstraction along with helper methods for most resources that make it simpler to define resources.
• L3 constructs, also known as patterns, are the highest-level of abstraction, and are used to create entire AWS architectures for particular use cases in your application.

Amazon Bedrock is a fully managed service that makes high-performing foundation models (FMs) from leading AI startups and Amazon available for your use through a unified API. As far as AWS CDK is concerned, Bedrock does not support L2 constructs yet.

AWS Generative AI Constructs Library is an open-source extension of the AWS Cloud Development Kit (AWS CDK) and fills the gap by providing L2 and L3 constructs to help developers build generative AI solutions using pattern-based definitions for their architecture.

Let's get started!

Prerequisites

Make sure to have the following setup and configured: AWS CDK, Python, Docker.

Also, configure and set up Amazon Bedrock, including requesting access to the Foundation Model(s). The application uses Amazon Titan Text Embeddings model along with the Anthropic Claude 3 Sonnet as the LLM.

Deploy the CDK stack

Start by cloning the GitHub repo:

git clone https://github.com/abhirockzz/aws-cdk-generative-ai-rag
cd aws-cdk-generative-ai-rag

Make sure to use the docs folder to add the documents (PDFs etc.) that you want to be used as a source for QnA. These will be automatically uploaded to a S3 bucket that will serve as the data source for Knowledge Base. Here is the list of document types supported by Knowledge Base.

Activate virtual env and install dependencies:

python3 -m venv venv

source venv/bin/activate
pip install -r requirements.txt

Deploy the stack:

cdk deploy

It might take at least five minutes for the deployment to complete. Once done, verify the CloudFormation stack in the AWS Console.

Navigate to the Knowledge Base and sync the data source with the OpenSearch Serverless vector index.

Start asking questions

For QnA, you can use the API Gateway endpoint. Send queries via a POST request - make sure you add /query at the end of the API Gateway URL. I had uploaded the Amazon 2022 shareholder document and asked the following question:
"What is Amazon doing in the field of generative AI?"

export APIGW_URL=<enter URL>

curl -X POST -d 'What is Amazon doing in the field of generative AI?' $APIGW_URL/query

Below is the result returned by the Lambda function based on response from RetrieveAndGenerate API:

{
  "question": "what is amazon doing in the field of generative AI?",
  "response": "Amazon is investing heavily in Large Language Models (LLMs) and Generative AI. The company believes that Generative AI will transform and improve virtually every customer experience across its businesses. Amazon has been working on developing its own LLMs for a while now. Amazon Web Services (AWS) is democratizing Generative AI technology by offering price-performant machine learning chips like Trainium and Inferentia, so that companies of all sizes can afford to train and run their LLMs in production. AWS also enables companies to choose from various LLMs and build applications with AWS's security, privacy and other features. One example is AWS's CodeWhisperer, which uses Generative AI to generate code suggestions in real time, boosting developer productivity."
}

Delete the stack

Once you are done, don't forget to delete the stack, which in turn will delete all the components:

cdk destroy

CDK stack code walk through

Here is a quick walkthrough of the different components in the stack - refer to the complete code here.

The Knowledge Base along with the S3 bucket, uploaded document(s), and the data source configuration:

        kb = bedrock.KnowledgeBase(self, 'DocKnowledgeBase', 
            embeddings_model= bedrock.BedrockFoundationModel.TITAN_EMBED_TEXT_V1,                  
        )

        documentBucket = s3.Bucket(self, 'DocumentBucket')

        deployment = s3deploy.BucketDeployment(self, "DeployDocuments",
            sources=[s3deploy.Source.asset("docs")],
            destination_bucket=documentBucket
        )

        bedrock.S3DataSource(self, 'KBS3DataSource',
            bucket= deployment.deployed_bucket,
            knowledge_base=kb,
            data_source_name='documents',
            chunking_strategy= bedrock.ChunkingStrategy.FIXED_SIZE,
            max_tokens=500,
            overlap_percentage=20   
        )

The Lambda function (along with necessary IAM policies) that invokes the RetrieveAndGenerate API - refer to the code here.

        kbQueryLambdaFunction = _lambda.Function(
            self, 'KBQueryFunction',
            runtime=_lambda.Runtime.PYTHON_3_12,
            code=_lambda.Code.from_asset('lambda'),
            handler='app.handler',
            environment={
                'KB_ID': kb.knowledge_base_id,
                'KB_MODEL_ARN': 'arn:aws:bedrock:us-east-1::foundation-model/anthropic.claude-3-sonnet-20240229-v1:0',
            },
            timeout=Duration.seconds(15)
        )

        kbArn = f'arn:aws:bedrock:{Stack.of(self).region}:{Stack.of(self).account}:knowledge-base/{kb.knowledge_base_id}'

        # Create an IAM policy statement
        policy_statement = iam.PolicyStatement(
            actions=[
                "bedrock:Retrieve",
                "bedrock:RetrieveAndGenerate",
                "bedrock:InvokeModel"
            ],
            resources=[
                "arn:aws:bedrock:us-east-1::foundation-model/anthropic.claude-3-sonnet-20240229-v1:0",
                kbArn
            ]
        )

        kbQueryLambdaFunction.add_to_role_policy(policy_statement)

Finally, the API Gateway with a query endpoint:

        api = apigateway.LambdaRestApi(
            self, 'KBQueryApiGW',
            handler=kbQueryLambdaFunction,
            proxy=False
        )

        kb_query = api.root.add_resource('query')
        kb_query.add_method('POST')

Conclusion

The patterns defined in AWS Generative AI CDK Constructs are high level, multi-service abstractions of AWS CDK constructs that provide well-architected patterns for quickly defining solutions in code to create predictable and repeatable infrastructure. I showed you a Python example, but there is TypeScript support as well.

I also encourage you to check out the samples repository that includes a collection of functional use case implementations to demonstrate the usage of AWS Generative AI CDK Constructs.

Happy Building!