Know what's inside your AWS S3 bucket using Rekognition and Bedrock

Introduction

Managing images in an S3 bucket is a common use case.

However, manually inspecting each image to understand the bucket's contents can be cumbersome.

Automating the process to generate a comprehensive summary of the bucket's content can greatly enhance efficiency and insight into what is stored within the bucket.

This blog explores a solution that provides a meaningful summary of your S3 bucket's contents, offering a streamlined approach to managing and understanding stored data.

Overview on AWS Services Used

• AWS S3 - Amazon Simple Storage Service (Amazon S3) is an object storage service offering industry-leading scalability, data availability, security, and performance.

• AWS Rekognition - Amazon Rekognition automates image recognition and video analysis for your applications without machine learning (ML) experience.

• AWS Bedrock - Amazon Bedrock is a fully managed service that offers a choice of high-performing foundation models (FMs) from leading AI companies like AI21 Labs, Anthropic, Cohere, Meta, Mistral AI, Stability AI, and Amazon

Approach

I have used boto3 library to interact with the AWS Services.

1. List Buckets and Objects in S3:

• List all the buckets available in Amazon S3. Once a user selects a specific bucket, you retrieve the objects stored inside that bucket.

2. Image Recognition with Amazon Rekognition:

• All images retrieved from the selected bucket are sent to AWS Rekognition.
• Rekognition identifies labels and provides predictions about the content of these images.

3. Using Predicted Labels as Context:

• Create a prompt for the LLM to summarize the contents of the bucket with the predicted labels of each image as the context.

4. Summarization Using Amazon Bedrock:

• Using Amazon Bedrock (model name = anthropic.claude-3-haiku-20240307-v1:0) to generate a summary.
• Provide the context (predicted labels + prompt) to LLM, to summarize the content of the bucket.

5. Returning the Summarized Response:

• The summarized response generated by the language model is returned to the user.

Things to know

• AWS Rekognition: Using AWS Rekognition for image analysis may incur costs based on the number of images processed.
• Amazon Bedrock: Accessing and utilizing the language model (LLM) within Amazon Bedrock for summarization may also involve costs. These costs can vary depending on the usage and the specific LLM model selected. Note, we have to request for model access in bedrock console.