Let's Build An Easy Computer Vision Web App with Streamlit & Streamlit-webrtc

Streamlit

It is an amazing Open-Source framework to create web apps with python with few lines of code. So, now instead of creating HTML pages and Css sheets for designing web apps, we have a tool Streamlit(framework). Which can ease all this work and can compile all the designing and the app functionality in one single piece of code script. So, let's dive deep to learn the functionalities of this tool. You can know more about this amazing framework by referring to this link - Streamlit documentation

Prerequisites

Before moving ahead you need to install following packages.

• streamlit

% pip install streamlit

• opencv-python

% pip install opencv-python

• streamlit-webrtc

% pip install streamlit-webrtc

• av

% pip install av

You're ready to build the app 🥳

We will first print the "Hello world" to launch our app.

Steps To Follow:

• Create an empty folder on your system.

• Move to that directory.
  

% cd /usr/include directory

• Create an empty app.py file in the same directory.

• Create a Virtual Environment in the same directory.(optional)
  

% python -m venv venv

• Activate the Virtual Environment.(macOS)

% . /venv/bin/activate

• Move to empty app.py file and write the following code and save the file.

import streamlit as st
st.write("Hello, world")

• Run the following command in the same directory to run the app.

% streamlit run app.py

This will boot up the streamlit server and the app will open in your browser.

OUTPUT

Streamlit_Webrtc

streamlit_webrtc Package is for Real-time Video/Audio Streaming.

Updating the app.py file to add the title and Video streaming component streamlit_webrtc to it. We will remove the "Hello World" from our app as that was just an example to show the functioning of our app.

• Importing all the pacakges required.
• Adding a title to the app- "Computer Vision Streamlit application"
• calling the webrtc_streamer() class imported from streamlit_webrtc for real-time video streaming.

import streamlit as st
from streamlit_webrtc import webrtc_streamer
st.title("Computer Vision Streamlit application")
webrtc_streamer(key="demo")

OUTPUT

When you will click on the "start" button, it will ask for a video and audio capturing permission. After you grant the permission it will show you live video streaming.

Let's add Video Processing component to the app using OpenCV.

We will update the app.py file to apply a filter using OpenCV on the video captured.

• In this tutorial we will use an Image processing filter of OpenCV Library called canny edge detection filter.
  You are free to use any other filter as per your choice.

• We will first create a class name VideoProcessor and define a function inside it, name recv(). This will be a callback function which will receive an input from the frame and return an output to the frame.

• The webrtc_streamer() can take a class object with .recv() function as its video_processor_factory argument.

• The argument of .recv() is the image frame from the webcam input video captured. we will convert it into a NumPy array with frame.to_ndarray() command.

-The return is from .recv() which will display on the screen with this command av.VideoFrame.from_ndarray(image, format="bgr24")

• Upadte the app.py file with following code.

import av
import cv2
import streamlit as st
from streamlit_webrtc import webrtc_streamer
st.title("Computer Vision Streamlit application")
class VideoProcessor:
    def recv(self, frame):
        image = frame.to_ndarray(format="bgr24")

        image = cv2.cvtColor(cv2.Canny(image, 100, 200), cv2.COLOR_GRAY2BGR)

        return av.VideoFrame.from_ndarray(image, format="bgr24")
webrtc_streamer(key="demo", video_processor_factory=VideoProcessor)

OUTPUT

wohhooo!🥳 We are ready with this app

Deployment

For deploying the app on Streamlit Cloud check out this Documentation.