INTRODUCTION:
Imagine a system that can analyze live video feeds in real time, interpret scenes, and respond intelligently to questions about the environment, just like a virtual assistant with eyes. This is the potential of combining cutting-edge technologies like OpenCV for video processing and Google's Gemini vision model, leveraging its latest "gemini-1.5-flash-latest" model.
In this article, I will guide you through building a Real-Time Object Detection System that uses live video streaming and AI-powered scene analysis to deliver insightful, context-aware responses. We'll deploy the application on AWS EC2, setting the stage for scalability and real-world use while employing Github Actions for automated CI/CD, ensuring a seamless update pipeline.
By the end of this tutorial, you'll have a fully functional AI-powered system ready for deployment, with the confidence to expand and customize it for various use cases.
PROJECT STRUCTURE
project/
├── app.py # Flask application code
├── requirements.txt # Python dependencies
├── templates/
│ └── index.html # Frontend UI
└── .env # Environment variables (API keys, etc.)
Core Components
A. Real-Time Video Capture (OpenCV)
The WebcamCapture class in app.py handles video streaming:
self.stream = cv2.VideoCapture(0) # Open the default webcam
This ensures efficient, thread-safe frame capture and processing.
B. AI-Powered Object Detection (Google Gemini)
Using the Gemini model, we analyze frames for real-time scene understanding:
self.model = ChatGoogleGenerativeAI(model="gemini-1.5-flash-latest")
response = self.chain.invoke({"prompt": prompt, "image_base64": image_base64})
C. Flask Backend
The Flask application provides endpoints for video streaming, AI queries, and system status checks:
/video_feed: Streams live video.
/process_query: Handles AI-powered analysis based on user input and video frames.
D. Frontend UI
The index.html file provides a responsive web interface for interacting with the system. It captures user queries and displays real-time AI responses.
PREREQUISITES
- An AWS account.
- A registered domain name (e.g., example.com).
- A Google Cloud Account or Open AI account
- GitHub actions configured in your repository.
- Basic knowledge of SSH and Linux command-line tools.
APPLICATION DEPLOYMENT
Step 1: Clone the Repository, Generate the Google Gemini API & Push the application files to Github
A. Clone the repository
$ git clone https://github.com/Abunuman/Real-Time-ODS.git
$ cd Real-Time-ODS
B. Generate your API key and add to a .env file
i. Create a .env file either manually from the options available on the left-hand side of your text editor (I used VScode)
OR
On the terminal, run:
$ touch .env
Then add these in the .env
GOOGLE_API_KEY=your_google_api_key
OPENAI_API_KEY=your_openai_api_key
FLASK_DEBUG=True
ii. Log into Google Cloud and follow these steps to generate your API key.
a. Navigate to the API & Services Section
b. Click on Credentials then follow the other steps below
Create Credentials > API Key , then the API Key is generated.
Remember to note the name of your API key. You can also give it a name during the process.
Copy the API Key generated, go back to your .env file and replace your_google_api_key with the key you just copied.
c. Enable Gemini API
Search for Gemini API and click on ENABLE
Confirm that your API Key is under the METRICS and Credentials section under the Enabled Gemini API.
iii. Create a .gitignore file and add .env to the file so that it is not pushed to github.
N.B.: Standard practice is to ensure that secrets and environment variables are not exposed to the public. Hence the need for a .gitignore to ignore files added therein while pushing to Github.
C. Push to Repository.
i. Create a Github repository with the application name and follow the commands below to push to github
$ git init
$ git add .
$ git commit -m "first commit"
$ git branch -M main
$ git remote add origin https://github.com/Abunuman/repository-name.git
git push -u origin main
N.B: change repository-name to your repository name
Step 2: Setting Up AWS EC2 Instance
i. Launch an EC2 Instance
- Use the AWS Management Console to launch an EC2 instance (e.g., Ubuntu 22.04).
- Select an instance type (e.g., t2.micro for free tier users).
- Create and download a key pair (.pem file) for SSH access.
Create a new key pair or use an existing one.
If you are creating a new key pair, click on create key pair and give it a name of your choice.
Select Key Pair type as RSA
File format as .pem
Then Create the key
The key pair is automatically downloaded to your system.
- Configure Security Groups
Allow the following inbound rules:
a. HTTP (port 80): For serving your application.
b. HTTPS (port 443): For secure access.
c. SSH (port 22): For access management.
- Click on Launch instance and allow the instance to be fully launched.
Now your instance is ready to use once the status shows “Running”.
ii. Configure the key pair (.pem key) for SSH access
For Mac book users or Linux users with bash terminal, configure your key pair for SSH access thus:
a. Open the downloaded .pem key using VScode or Xcode
b. On your terminal, navigate to the .ssh directory from the root directory(~)
$ cd .ssh
c. Create a .pem file in the .ssh directory using nano or vim text editors; I will be using nano in this tutorial.
Install nano if you don't have it installed.
For macbook users
$ brew install nano
For linux users
$ sudo apt install nano
Having installed it, create the .pem file in the .ssh directory using nano.
Ensure the file to be created bears the exact name of your .pem file.
$ sudo nano name_of_pem.pem
Then copy the already opened .pem file and paste in the .pem to be created in the .ssh directory.
Press Ctrl X, then Y , then Enter to save.
d. Change the .pem file permission
$ chmod 400 name_of_pem.pem
iii. Access the Instance - SSH into your EC2 instance:
Click on the Instance ID . Once the instance is in the running state, select on the connect option
Once you are at the Connect page , Go to SSH Client
Then copy the last command on the page that looks like this:
ssh -i path/to/key.pem ubuntu@<ec2-public-ip>
Paste this on your terminal and press enter. You should connect seamlessly.
For Windows Users
- Windows Setup
Open CMD on your windows machine
Locate desired directory where .pem file is stored
Ideally from this directory , we can run the copied ssh command and we should be able to connect to EC2.
However, sometimes we get a security permissions error when we run the ssh command.
We have to change the permissions to the .pem file.
For that follow the steps below:
Locate the .pem file folder , right click on the file and select properties
Go to Security tab
Go to Advanced tab
Click Disable inheritance
This Advance options also shows other user having all permissions to .pem file. Remove permission for all other users
Add the user with which you are trying to connect to EC2 if not already present in the user list.
Enable all permissions for this user.
Ideally with these steps, you should not encounter an error.
Run the SSH command from CMD prompt
Once the permissions are fixed , prompt will successfully connect to EC2
Now, you have successfully completed the steps and you can run commands from windows CMD on EC2 instance.
iv.Install Dependencies - Update the package list and install necessary packages:
Having connected to your EC2 instance via SSH, install dependencies on EC2.
On your connected terminal, run the following commands:
$ sudo apt update
$ sudo apt install -y python3 python3-pip nginx
Check the version of python3 installed, ensure its 3.12
python3 --version
Step 3: Configuring GitHub Actions for CI/CD
A. Set up Github Actions Environment Secrets
Configure your AWS IAM user secrets and environment variables needed for the project.
Set GitHub Secrets in your repository, navigate to Settings > Secrets and variables > Actions, and add:
EC2_USERNAME (e.g., ubuntu).
EC2_HOST (your EC2 public IP or domain).
EC2_SSH_KEY (path to your private key file).
i. Navigate to Settings in your repository
ii. Click on Secrets and Variables, then Actions
iii. Add your Secrets and Variables like below
B. Create a Workflow File in your repository, add a .github/workflows/main.yml file:
Navigate to Actions under your repository and click on set up a workflow yourself
Then add this to the space provided
name: Deploy to AWS
on:
push:
branches: [main]
jobs:
deploy:
runs-on: ubuntu-latest
steps:
# Check out the repository
- uses: actions/checkout@v4
# Set up Python environment
- name: Set up Python 3.12
uses: actions/setup-python@v5
with:
python-version: "3.12"
# Configure AWS credentials
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v4
with:
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
aws-region: ${{ secrets.AWS_REGION }}
# Copy files to EC2 instance
- name: Transfer files to EC2
uses: appleboy/scp-action@master
with:
host: ${{ secrets.EC2_HOST }}
username: ${{ secrets.EC2_USERNAME }}
key: ${{ secrets.EC2_SSH_KEY }}
source: "app.py,requirements.txt,templates/index.html"
target: "/home/${{ secrets.EC2_USERNAME }}/app"
# SSH to EC2 and set up the app
- name: Deploy to EC2
uses: appleboy/ssh-action@master
with:
host: ${{ secrets.EC2_HOST }}
username: ${{ secrets.EC2_USERNAME }}
key: ${{ secrets.EC2_SSH_KEY }}
script: |
# Create and navigate to the app directory
cd /home/${{ secrets.EC2_USERNAME }}/app
# Ensure system dependencies are installed
sudo apt-get update
sudo apt-get install -y python3-venv python3-pip git libopencv-dev python3-opencv portaudio19-dev ffmpeg
# Setup Python virtual environment
if [ ! -d "venv" ]; then
python3 -m venv venv
fi
# Activate virtual environment and install dependencies
source venv/bin/activate
pip install --upgrade pip
pip install -r requirements.txt
# Ensure templates directory exists and move files if needed
mkdir -p templates
# Create or update systemd service
sudo tee /etc/systemd/system/flask_app.service << EOF
[Unit]
Description=Flask App
After=network.target
[Service]
User=${{ secrets.EC2_USERNAME }}
WorkingDirectory=/home/${{ secrets.EC2_USERNAME }}/app
Environment="PATH=/home/${{ secrets.EC2_USERNAME }}/app/venv/bin"
ExecStart=/home/${{ secrets.EC2_USERNAME }}/app/venv/bin/python app.py
[Install]
WantedBy=multi-user.target
EOF
# Reload and restart the service
sudo systemctl daemon-reload
sudo systemctl enable flask_app
sudo systemctl restart flask_app
Having done this, run the CI/CD and check your terminal connected to EC2 to check if the files are now moved to EC2.
Step 4: Acquiring an SSL Certificate with AWS ACM
A. Request a Certificate
Navigate to AWS Certificate Manager in the AWS Console.
Click Request a certificate and choose Request a public certificate.
B. Enter your domain name (e.g., example.com)
C. Choose DNS validation and complete the request.
D. Validate the Certificate
ACM will provide a CNAME record.
Add this record in your domain’s Route 53 hosted zone.
E. Verify the Status
Once validated, the certificate status will change to Issued.
Step 5: Managing a Custom Domain with AWS Route 53
A. Navigate to AWS Route 53 in the AWS Console - Create a Hosted Zone for your domain
B. Update Domain Name Servers
Copy the nameservers (NS) from the hosted zone and update the NS records in your domain registrar’s settings.
C. Add an A Record
Create an A record pointing to your EC2 instance's public IP.
e.g realtime.example.com (realtime as sub-domain)
D. Add a CNAME for SSL
Add a CNAME record from ACM for DNS validation if not already done.
This can be created during your SSL certificate generation by clicking on the CNAME option and it redirects you to AWS Route53 and its created for the domain.
You can also create it under AWS Route 53 by going through similar steps followed in creating an A record.
Step 6: Configure NGINX for reverse proxy
Configure NGINX for reverse proxy
A. Edit nginx sites available for flask app
Navigate to /etc/nginx/sites-available/flask_app:
$ sudo nano /etc/nginx/sites-available/flask_app
B. Add this configuration
map $http_upgrade $connection_upgrade {
default upgrade;
'' close;
}
# HTTP redirect server block
server {
listen 80;
server_name your_a_record_domain.com;
# Only redirect if not already HTTPS
if ($http_x_forwarded_proto != 'https') {
return 301 https://$server_name$request_uri;
}
location / {
proxy_pass http://localhost:5000;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
# Increased timeouts for streaming
proxy_read_timeout 3600s;
proxy_send_timeout 3600s;
}
}
# Main HTTPS server block
server {
listen 443;
server_name your_a_record_domain.com.com;
# Increase timeout settings
proxy_connect_timeout 600;
proxy_send_timeout 600;
proxy_read_timeout 600;
send_timeout 600;
# Prevent timeout for video streaming
keepalive_timeout 650;
keepalive_requests 10000;
client_max_body_size 50M;
location /video_feed {
proxy_pass http://localhost:5000;
proxy_buffering off;
proxy_cache off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_http_version 1.1;
proxy_set_header Connection '';
proxy_read_timeout 24h;
proxy_send_timeout 24h;
proxy_connect_timeout 24h;
}
location /process_frame {
proxy_pass http://localhost:5000;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_cache off;
proxy_buffering off;
}
location / {
proxy_pass http://localhost:5000;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
}
C. Restart nginx and flask app
Run:
$ sudo systemctl restart nginx
$ sudo systemctl restart flask_app
Step 7: Create an Application Load Balancer
To secure your application with HTTPS using an SSL certificate from AWS Certificate Manager (ACM), you need to link the SSL certificate to your domain via an Application Load Balancer (ALB) and configure your domain in Route 53.
You can create the Load Balancer thus:
A. Set Up an Application Load Balancer (ALB)
Go to the AWS Management Console and navigate to EC2 > Load Balancers.
B. Click Create Load Balancer and choose Application Load Balancer.
Step-by-Step Configuration:
Name: Provide a name (e.g., realtime-alb).
Scheme: Choose Internet-facing.
C. Listeners:
Add two listeners:
- Port 80 (HTTP) → Target Group (Redirect to HTTPS).
- Port 443 (HTTPS) → SSL Certificate.
- Availability Zones: Select the VPC and all subnets for redundancy.
- Configure HTTPS Listener and Attach SSL Certificate
In the Listeners section during ALB creation:
For Port 443 (HTTPS):
Choose SSL Certificate from AWS Certificate Manager (ACM).
Select the validated certificate for your domain (e.g., example.com).
Forwarding Rules:
Under Actions, ensure the HTTPS listener forwards traffic to a Target Group that includes your EC2 instance.
D. Set Up a Target Group
Go to Target Groups in the EC2 dashboard.
- Create a target group:
Name: e.g, realtime-tg.
Target Type: Instances.
Protocol: HTTP (port 80).
- Register your EC2 instance:
Select your running EC2 instance and add it to the target group.
- Verify the health checks:
ALB will perform periodic checks to ensure your application is running.
Use HTTP and / for health checks.
E. Redirect HTTP to HTTPS (Optional but Recommended)
In the Load Balancer Listeners section:
Add a rule for Port 80 to redirect all traffic to HTTPS (Port 443).
Set the redirection response to 301 Moved Permanently.
Example Rule:
Condition: All requests
Action: Redirect to HTTPS (port 443).
F. Update Route 53 to Point to the Load Balancer
Go to Route 53 and navigate to your hosted zone.
Click on your A Record and edit thus:
Alias: Enable Alias.
Target: Select your Application Load Balancer from the dropdown list.
Save the record.
Finally, you can restart the flask app and nginx on your connected terminal, and check the status afterwards
$ sudo systemctl restart nginx
$ sudo systemctl restart flask_app
$ sudo systemctl status nginx
$ sudo systemctl status flask_app
Final Test
Visit your domain e.g https://real.example.com to access your deployed application securely. You should see the real-time object detection system up and running!
CONCLUSION
In this tutorial, we embarked on a comprehensive journey to build and deploy a real-time object detection system that seamlessly integrates OpenCV for live video capture and Google's Gemini vision model for intelligent scene analysis.
From configuring the application locally to deploying it securely on AWS EC2 with a custom domain and SSL, we covered every essential step to transform your idea into a functional and scalable solution.
This project highlights the power of combining cutting-edge technologies like Flask, OpenCV, and AI to solve real-world problems while ensuring best practices for cloud deployment.
By following these steps, you've not only deployed a robust AI-powered system but also ensured scalability, security, and efficient CI/CD pipelines.
Top comments (0)