In this post, I'll walk you through how to create a voice-based AI career counseling mentor that speaks Hindi using Twilio for voice calls and Groq's LLaMA models for generating AI responses. This bot will act as a friendly and knowledgeable mentor, guiding students through career-related queries in Hindi. We'll also use ngrok to expose our local Flask server to the internet.
Overview of the Tech Stack
- Twilio: Handles voice calls and speech recognition.
- Groq's LLaMA Models: Powers the AI mentor with natural language understanding and response generation.
- Flask: A lightweight Python web framework to handle HTTP requests and responses.
- ngrok: Exposes your local Flask server to the internet so Twilio can interact with it.
How It Works
-
Incoming Call: Twilio forwards the call to your Flask app's
/voiceendpoint. -
Speech Recognition: The app uses Twilio's
Gatherto capture the user's speech. - AI Response: The app sends the speech text to Groq's LLaMA model, which generates a response in Hindi.
- Text-to-Speech: Twilio's Polly.Aditi converts Hindi text into speech and plays it back.
- Continuous Conversation: The bot continues the conversation by prompting the user for more input.
Step-by-Step Implementation
1. Set Up Your Environment
Before diving into the code, ensure you have the following:
- Python 3.10+ installed.
- A Twilio account with a phone number capable of making voice calls.
- A Groq API key (sign up at Groq).
- Install the required Python libraries:
pip install flask twilio groq
2. Write the Code
a. Initialize Flask and Dependencies
from flask import Flask, request, Response, send_from_directory
from twilio.twiml.voice_response import VoiceResponse, Gather
from twilio.rest import Client
from groq import Groq
import os
# Initialize Flask
app = Flask(__name__)
# Twilio Credentials
TWILIO_ACCOUNT_SID = os.getenv('TWILIO_ACCOUNT_SID')
TWILIO_AUTH_TOKEN = os.getenv('TWILIO_AUTH_TOKEN')
TWILIO_PHONE_NUMBER = os.getenv('TWILIO_PHONE_NUMBER')
MY_PHONE_NUMBER = os.getenv('MY_PHONE_NUMBER')
twilio_client = Client(TWILIO_ACCOUNT_SID, TWILIO_AUTH_TOKEN)
# Initialize Groq client
client = Groq(api_key=os.getenv('GROQ_API_KEY')) # Add your API key here
ngrok_url = 'https://your-ngrok-url.ngrok.io' # Replace with your ngrok URL
messages = [
{
"role": "system",
"content": """You are a highly knowledgeable and professional mentor female voice assistant who can solve any query regarding to career choice. Provide insightful, personalized, and practical guidance to students regarding their career and academic path. Focus on delivering clear, concise, expert advice using natural, conversational language suitable for Hindi translation and speech. Be encouraging and supportive. Keep responses short and to the point. Start with the greeting and flow should be human-like. Behave like you are his friend and guide."""
}
]
b. Handle Incoming Voice Calls
@app.route("/voice", methods=['POST'])
def voice():
"""Handles incoming voice calls from Twilio."""
response = VoiceResponse()
response.say("Namaste, aap kaise hai?", voice="Polly.Aditi", language="hi-IN")
gather = Gather(input="speech", action="/process_voice", method="POST", language="hi-IN", speechTimeout="auto")
gather.say("कृपया अपनी समस्या बताएं।", voice="Polly.Aditi", language="hi-IN")
response.append(gather)
return Response(str(response), content_type='text/xml')
This function greets the user in Hindi and prompts them to describe their career-related query using Twilio's Gather verb for speech input.
c. Process User Speech and Generate AI Response
@app.route("/process_voice", methods=['POST'])
def process_voice():
"""Processes user Hindi speech input, generates a response, and returns a spoken reply."""
response = VoiceResponse()
speech_text = request.form.get("SpeechResult", "")
if not speech_text:
response.say("मुझे आपकी आवाज़ सुनाई नहीं दी, कृपया फिर से बोलें।", voice="Polly.Aditi", language="hi-IN")
response.redirect("/voice") # Retry input
return Response(str(response), content_type='text/xml')
print(f"User said: {speech_text}")
# Generate AI response using Groq
ai_response = generate_response(speech_text)
print(f"AI response: {ai_response}")
# Stream the response back to the user
response.say(ai_response, voice="Polly.Aditi", language="hi-IN")
# Continue the conversation
gather = Gather(input="speech", action="/process_voice", method="POST", language="hi-IN", speechTimeout="auto")
response.append(gather)
return Response(str(response), content_type='text/xml')
d. Generate AI Response Using Groq
def generate_response(prompt):
"""Uses Groq to generate AI responses in Hindi based on conversation context."""
messages.append({"role": "user", "content": prompt})
try:
ai_response = client.chat.completions.create(
model="llama-3.3-70b-versatile", # Replace with your preferred Groq model
messages=messages,
max_tokens=32768,
temperature=0.5,
n=1
)
res = ai_response.choices[0].message.content.strip()
messages.append({"role": "assistant", "content": res})
return res
except Exception as e:
print(f"Error generating response: {e}")
return "मुझे क्षमा करें, मैं आपकी सहायता नहीं कर सकता।"
3. Set Up ngrok
Run:
ngrok http 5000
Copy the HTTPS URL provided by ngrok and update ngrok_url in the Flask app.
4. Configure Twilio
- Log in to your Twilio console.
- Go to Phone Numbers > Manage > Active Numbers and select your Twilio phone number.
- Under the Voice & Fax section, set the Webhook for incoming calls to
https://your-ngrok-url.ngrok.io/voice.
Conclusion
By combining Twilio and Groq's LLaMA models, you can create a powerful Hindi-speaking AI career counseling mentor. This project demonstrates the potential of AI in providing personalized guidance in regional languages, making it accessible to a wider audience.
Let me know if you have any questions or suggestions!
Top comments (1)
Unique Idea and Great Implementation!