Building an Answer Scoring Agent with Lyzr ChatAgent and Streamlit

In the educational landscape, assessing students' answers accurately and efficiently is crucial for effective learning outcomes. To streamline this process, leveraging technology can be immensely beneficial. In this blog post, we'll explore how to build an Answer Scoring Agent using Streamlit, a popular Python framework for building interactive web applications, and Lyzr, a powerful ChatBot tool.

Assessing answers manually can be time-consuming and subjective. By automating this process, educators can save time and ensure more consistent evaluations. 

Key Features
PDF Upload: Users can upload a PDF file containing questions.
Manual Input: Users can input questions manually if they don't have a PDF file.
Answer Evaluation: Lyzr evaluates the user's answer based on predefined criteria.
Dynamic Grading: The scoring criteria can be customized based on the question type and context.

Setting Up the Environment

pip install streamlit lyzr
import os
import streamlit as st
from lyzr import ChatBot
from utils.llm_calling import llm_calling
import shutil

os: For interacting with the operating system (file operations).
streamlit as st: For creating the web application interface.
ChatBot from lyzr: For handling question-answering from a PDF document.
llm_calling: For using a language model to generate grading criteria and grades (specific implementation not shown).
shutil: For removing files and directories.
Image from PIL: For displaying an image.

Input TextBooks:

uploaded_file = st.sidebar.file_uploader("Choose PDF file", type=["pdf"])
default = st.sidebar.button("Use Default")

Provides instructions for uploading a textbook or using a sample chemistry book.
Specifies the topics covered in the sample book.

File Upload and Handling:

def remove_existing_files(directory):
    for filename in os.listdir(directory):
        file_path = os.path.join(directory, filename)
        try:
            if os.path.isfile(file_path) or os.path.islink(file_path):
                os.unlink(file_path)
            elif os.path.isdir(file_path):
                shutil.rmtree(file_path)
        except Exception as e:
            print(e)

data_directory = "data"
os.makedirs(data_directory, exist_ok=True)
remove_existing_files(data_directory)

if uploaded_file is not None:
    # Save the uploaded PDF file to the data directory
    file_path = os.path.join(data_directory, uploaded_file.name)
    with open(file_path, "wb") as file:
        file.write(uploaded_file.getvalue())
    # Display the path of the stored file
    st.success(f"File successfully saved")

Defines a function remove_existing_files to clear any existing files in a directory.
Creates a directory named "data" to store uploaded files.
Creates a file uploader in the sidebar with options to upload a PDF or use a default file ("lech201.pdf").
If a file is uploaded, saves it to the "data" directory and displays a success message.

Retrieve File Paths:

def get_all_files(data_directory):
    # List to store all file paths
    file_paths = []

    # Walk through the directory tree
    for root, dirs, files in os.walk(data_directory):
        for file in files:
            # Join the root path with the file name to get the absolute path
            file_path = os.path.join(root, file)
            # Append the file path to the list
            file_paths.append(file_path)

    return file_paths

Defines a function get_all_files to retrieve all file paths within the "data" directory.

Create a Q&A Chatbot:

def bookqabot(file_path):
    qa_bot = ChatBot.pdf_chat(
        input_files=[file_path],
    )

    return qa_bot

defines a function bookqabot that takes a PDF file path as input and creates a Lyzr chatbot for Q&A based on the PDF content.

Generate Grade:

def generate_grade(question, answer, ref_answer):
    res = llm_calling(user_prompt=f"For a 2 marks quetsion on {question} what should be the gradding criteria",
                      system_prompt="You are a grading criteria generator for questions",
                      llm_model="gpt-4-turbo-preview")
    prompt1 = f"""
                Give Grade for Answer for Question Based On Criterias and Reference answer.
                Question: {question}
                Answer: {answer}
                Criterias: {res}
                Reference Answer:{ref_answer}
                [!important] only give grades nothing apart from that

                Grades: 2/2
                """

    grade = llm_calling(user_prompt=prompt1, system_prompt="You are a giving grade to answers.",
                        llm_model="gpt-4-turbo-preview")
    return grade

Defines a function generate_grade that uses a language model to:
Generate grading criteria based on a question.
Generate a grade for an answer based on the question, criteria, and a reference answer.

Combine Answer and Grade:

def combine_answer(question,answer,qa):
    qa_res = qa.chat(question)
    grade = generate_grade(question, answer, qa_res)
    st.markdown(grade)

paths = get_all_files(data_directory)

file_path = "default_file_path"

Defines a function combine_answer that calls the chatbot for an answer and then calls generate_grade to grade the provided answer.
Displays the grade using Streamlit's markdown function.

Main Application Logic:

if paths:
    for path in paths:
        qa = bookqabot(path)
        question = st.text_input("Enter your question: ")
        answer = st.text_area("Enter Your Answer: ")
        if st.button("Grade"):
            combine_answer(question,answer,qa)
else:
    if st.session_state.get('button') != True:
        st.session_state['button'] = default
    if st.session_state['button'] == True:
        qa = bookqabot(file_path)
        question = st.text_input("Enter your question: ")
        answer = st.text_area("Enter Your Answer: ")
        if st.button("Grade"):
            combine_answer(question,answer,qa)

Retrieves file paths from the "data" directory.
If paths are found:
Iterates through each path:
Creates a chatbot for the PDF file.
Collects a question and answer from the user.
Displays the grade if the "Grade" button is clicked.
If no paths are found (no user uploaded file):
Checks for the "Use Default" button state.
If the default button is activated:
Creates a chatbot for the default PDF file.
Collects a question and answer from the user.
Displays the grade if the "Grade" button is clicked.

In this blog post, we've demonstrated how to build an Answer Scoring Agent using Streamlit and Lyzr. By automating the answer evaluation process, educators can save time and ensure more consistent assessments. This application can be further extended with additional features such as user authentication, feedback generation, and performance analytics.

try it now: https://lyzr-answer-scoring.streamlit.app/

For more information explore the website: Lyzr