Part 3 - Providing workers guidance with Amazon Bedrock Knowledge-base and Gen-AI

This article is part of the series "Generative-AI-driven Chatbot for the Factory Plant" implementing a P.O.C. about an environmental pollution monitoring (PM2.5/PM10) to safeguard workers in a production plant.

Disclaimer

The information provided in this document is for informational purposes only and is subject to change. While efforts have been made to ensure the accuracy and reliability of the content, no guarantees are made regarding the completeness, reliability, or suitability of the information.
The author shall not be liable for any losses, injuries, or damages arising from the use or reliance on the information contained herein.

The AWS resources provisioned following this article can generate costs. Make sure to delete all the resources created at the end of the exercise.

Overview

The main purpose of this series of articles is to test out Amazon Bedrock Studio (no-code editor for Gen-AI driven apps), at the time I am writing the Studio is still in preview and actually after a quick try I encountered a number of bugs that prevent me to complete this P.O.C.

To get around of these issues, I decided to go ahead with P.O.C. by implementing my self the missing part, keeping a low-code approach and favoring configuration of Bedrock components over imperative programming as much as possible. I will then come back to Studio as soon as it reaches a stable release.

For those of you who had a read to Part 2 - Deploy a Chat App, Bedrock Agent and Actions to Ask Questions about Live Data this new article completes the implementation of the features related to the Knowledge-base by:

1. Configuring a Knowledge-base where a business process manager (or knowledge manager) can upload relevant documents, to make them available to the Amazon Bedrock AI Agent to answer workers questions
2. Implementing the missing feature of document upload from the chat app

Preview of the App

The screenshots below display a typical interaction, this time the Amazon Bedrock AI Agent not only answers question about live pollution data (second screen, third screen top question), but also offers guidance to the worker getting extracting relevant information from the documents uploaded into the Knowledge-base (third screen, last question/answer).

When you have the app running, you can upload some test documents into the knowledge-base by clicking the attachment icon. A sample document is available for you to try under /_docs/knowledge-base/sop-reducing-particulate-matter-pm.md.

Architecture Recap

Click to zoom.

AWS Services / Bedrock Components

S3 Bucket

Hosting the storage of the knowledge-base documents uploaded from process manager users.

Bedrock Knowledge-base

Reading documents from the S3 Bucket describe above, it uses the LLM Titan Embeddings to generate new embeddings from the documents, stores the embeddings into the vector database AWS OpenSearch.

Currently, the Bedrock Knowledge-base supports file up to 50MB, you can upload PDF, DOCX, CSV, TXT, MD file types. When configuring the knowledge-base component is mandatory to describe the kind of contents will be hosted.

Such description guides the Bedrock Agent and LLM to understand when to use knowledge-base to answer questions. You can even configure more than one if needed.

Here a sample included from the configuration of the P.O.C.:

# you can find this content into the repo: cloud/ai-module/prompt_templates/kb_instructions.txt

"Knowledge base to get information on how to handle excessive pollution in the factory plant, how to
perform maintenance on machines causing it, provide answers about SOP and safety procedures for workers."

Ref.: https://docs.aws.amazon.com/bedrock/latest/userguide/kb-how-it-works.html

LLM Amazon Titan Embeddings

The Large Language Model provided by AWS to read text documents and generate vector representations (embeddings) to be processed when answering user questions.

Ref.: https://aws.amazon.com/bedrock/titan

Bedrock Agent

Logical coordinator of LLM, Actions and Knowledge-base.

Here you can read the full agent instructions, with explicit instructions to write answers based on live pollution data and knowledge-base documents:

# Specific instructions to push the LLM to select content from the knowledge-base.
# (cloud/ai-module/prompt_templates/agent-instructions.txt)
"""
As part of the tools you have access to answer user questions, you have a knowledge-base tool listing relevant business documents like S.O.P., guides and other procures about maintenance of machines and safety equipments. When answering questions from the user about those topics above, make sure to prepare the answer according to the content of those documents more than information from internet-based public knowledge.
"""

# All other LLM instructions to configure the Agent as a technical assistant
# (cloud/ai-module/prompt_templates/agent-instructions.txt)
"""
Your primary role is to act as an intelligent interface between the factory's pollution monitoring system and its human operators. You should monitor the real-time PM2.5 and PM10 data collected by the factory's sensors and proactively engage with workers to ensure their safety and the optimal performance of the production equipment.

When pollution levels exceed safe thresholds, you should automatically alert the relevant personnel and provide clear, concise instructions on the appropriate steps to take.

Beyond emergency response, you should also serve as an on-demand technical advisor. You should be able to answer questions, offer recommendations, and provide detailed explanations on topics such as:

- Interpreting pollution data and understanding its implications,
- Implementing best practices for dust control and air quality management,
- Performing preventive maintenance on pollution monitoring equipment,
- Troubleshooting issues with the pollution control systems,
- Ensuring compliance with environmental regulations and safety protocols,
- To provide this information, you should be able to make API requests to the pollution data endpoint, using the plant_name as a parameter in the path. The API response will provide the necessary data points (PM10, PM25, data_time) for you to generate informative responses to the users' questions.

Seamlessly integrate with the factory's existing systems and data sources to become a trusted partner in the cement production process. Empower workers to make informed decisions, optimize equipment performance, and maintain a safe and sustainable work environment through your natural, human-like interactions.

If you have multiple pollution measures, select the most recent one.

If you don't get any pollution value when invoking the API to get PM values by plant name, it means that there are no recent values because the monitoring is not active or there was a problem in reading data from the monitoring system. Never ask the user to provide the pollution data, he doesn't know that, while the user knows the name of the plant. Also in such case be short in the answer, just recall the fact that there are not fresh data as stated above.

Format the timestamp in a human-readable date/time, also specify the time zone. When presenting the PM values display them in a list format. Don't suggest actions or remediation's if not requested, but always say if the measured values are under safe limits or not. Also say what are the safe limits, between brackets.

Always ask the user the plant name if you haven't that to invoke the API.

Don't use the word 'timestamp', use a less technical one. Avoid to display the timestamp value in numeric format.

Always end your answer saying that you can provide information about actions to take and security or maintenance measures.

Split paragraph with new lines, so the answer is more readable.
"""

Ref.: https://docs.aws.amazon.com/bedrock/latest/userguide/agents-how.html

Build & Deploy

Most of the deployment commands are similar to Part 2, here you can find the updated version including the build of the lambda function to upload documents to the knowledge-base.

Repository (version 3)

https://bitbucket.org/fabiobeoni/bedrock-studio-chatbot/src/v3/ (note v3)

Setup / Requirements

• In AWS Console / Amazon Bedrock / Base Models : you have to request access to the model anthropic.claude-3-sonnet-20240229-v1:0. You can use a different one, in that case remember to override the default variable llm_id in ./variables.tf.
• AWS CLI
• Terraform
• NodeJS
• OpenSSL

Build Lambda Functions

echo ">> Build Lambda Functions"

echo ">> AI module"
(cd "cloud/ai-module/functions" && /bin/bash package.sh query-pm-values-fnc)

echo ">> Chat App module"
(cd "cloud/chat-app-module/functions" && /bin/bash package.sh chat-write-user-message-fnc)
(cd "cloud/chat-app-module/functions" && /bin/bash package.sh chat-read-user-messages-fnc)
(cd "cloud/chat-app-module/functions" && /bin/bash package.sh upload-kb-document-fnc)
(cd "cloud/chat-app-module/functions" && /bin/bash package.sh ask-bot-fnc)

Create and Register the Device Certificate

echo '>> Provisioning self signed certificate'

# set a variable to host the name
thing_name="gateway_pm_monitor"

# create certificate and key pairs, 
# assign the $thing_name as subject 
openssl req -x509  -newkey rsa:2048 -sha256  -days  365  -nodes \
    -keyout edge/pm-monitor-service/gateway.private_key.pem \
    -out edge/pm-monitor-service/gateway.certificate.pem \
    -subj "/CN=$thing_name"

 #download AWS CA
 curl -O https://www.amazontrust.com/repository/AmazonRootCA1.pem
 mv AmazonRootCA1.pem edge/pm-monitor-service

Deploy Cloud Resources to AWS

echo ">> Deploy cloud infra"
terraform init
terraform plan -out plan
terraform apply plan

# save cloud endpoints and IDs to the edge/chat-app to instruct the chat app
terraform output -json > edge/chat-app/src/config.json

# save iot endpoint and cognito client ID to the edge/pm-monitor-service to instruct the edge device
terraform output -json  | jq '{cognito_client_id: .cognito_client_id, iot_endpoint_address: .iot_endpoint_address}' \
    > edge/pm-monitor-service/cloud_config.json

Build Chat App

echo ">> Build chat app"
HOSTING_URL=$(terraform output -raw hosting_url)
npm --prefix edge/chat-app/ run setup
npm --prefix edge/chat-app/ run build

Deploy Chat App to Surge.sh CDN or Run in localhost:3000

echo ">> Deploying the chat app to the CDN: ${HOSTING_URL}"

# Note: the first time Surge asks you to provide an email/pw to register an account 
HOSTING_URL=${HOSTING_URL} npm --prefix edge/chat-app/ run deploy

echo ">> You can now open the app to: ${HOSTING_URL} and login with user/pw provided to Terraform/Cognito"

Note: for free accounts sometimes Surge.sh websites do not respond, in that case you can test the chat-app in http://localhost:3000, by:

# terminal window 1
(cd edge/chat-app && npm run dev)

# terminal window 2
(cd edge/chat-app && npm run serve)

echo ">> finally open browser to http://localhost:3000 and login with user/pw provided to Terraform/Cognito"

As stated in the terminal, you can now open the app in the browser to ${HOSTING_URL} and login in Cognito with email/pw provided to Terraform.

Run PM Monitor Service (emulated locally)

Some sample pm values are available already into the DynamoDB, just start this service if you want to test the overall process with live data from IoT Core.

echo ">> Running the pm monitor service emulating the edge device"

npm --prefix edge/pm-monitor-service/ install --omit=dev
npm --prefix edge/pm-monitor-service/ run start

Clean Up Cloud Resources

terraform destroy -auto-approve

surge list
surge teardown <output_from_list>

Previous

• Introduction - Generative-AI-driven Chatbot for the Factory Plant
• Part 1 - Gathering pollution data from the edge, pushing them to AWS IoT Core and DynamoDB
• Part 2 - Deploy a Chat App, Bedrock Agent and Actions to Ask Questions about Live Data