Async Interview Feedback Platform powered by AssemblyAI

This is a submission for the AssemblyAI Challenge: No More Monkey Business.

What I Built

In the modern job market, interviews are crucial, yet feedback is often inadequate or nonexistent. This lack of feedback not only frustrates candidates but also fails to provide them with actionable insights for improvement. To solve this, I built an Async Interview Feedback Platform powered by AssemblyAI, which leverages advanced AI transcription and summarization capabilities to deliver detailed interview reports, including candidate strengths, weaknesses, and recommendations.

The platform transforms the traditional interview process by recording interviews, transcribing them with near-perfect accuracy using AssemblyAI, and generating structured feedback reports with insights into technical, behavioral, and communication skills. By addressing the pain points of vague or absent feedback, my platform empowers both candidates and interviewers with actionable insights, improving overall hiring quality.

Demo

You can access a live demo of the platform here: Demo Link
Here are a few screenshots of the application in action:

Journey

The journey to creating this platform was an exciting blend of challenges and learning. Here's a breakdown of how I implemented it:

Idea Validation

The project idea stemmed from personal experience and feedback from peers who felt they never received meaningful feedback after interviews. I wanted to automate this feedback loop using AI, ensuring accuracy, consistency, and scalability.

Technology Stack

I used the following tools and services:

• AssemblyAI for transcription and summarization.
• Convex.dev for real-time updates and database management.
• Inngest for managing background workflows.
• Resend for email notifications.
• React for the frontend interface.
• Node.js for the backend API.

Implementation

1. Setting Up Transcription

The heart of the platform is AssemblyAI's transcription service. Here's how I integrated it:

const client = new AssemblyAI({ apiKey: process.env.ASSEMBLYAI_API_KEY });

async function transcribeRecording(audioUrl: string) {
  try {
    const transcript = await client.transcripts.transcribe({ audio_url: audioUrl });
    console.log('Transcription completed:', transcript.text);
    return transcript;
  } catch (error) {
    console.error('Error during transcription:', error);
    throw error;
  }
}

AssemblyAI’s API was not only easy to integrate but also incredibly accurate, delivering transcripts with minimal errors even in challenging audio conditions.

2. Generating Insights with AssemblyAI’s LEMUR

AssemblyAI’s LEMUR tool was instrumental in generating insightful summaries and structured feedback. I used it to craft a Markdown report template tailored to interview analysis:

const prompt = `
Objective: Summarize the candidate's key strengths and weaknesses from the transcript.
Additional Instructions: Provide actionable insights focused on technical skills and communication.

## Final Report
### Strengths
- List of key strengths.

### Weaknesses
- List of areas needing improvement.

### Recommendations
- Final recommendation and rationale.
`;

async function generateInsights(transcriptIds: string[]) {
  const response = await client.lemur.task({
    transcript_ids: transcriptIds,
    prompt,
    temperature: 0.4,
    max_output_size: 4000,
  });
  return response.reportInMarkdown;
}

3. Automating the Workflow with Inngest

To handle asynchronous workflows like transcription and report generation, I used Inngest, which ensured fault tolerance and retry mechanisms:

export const generateReport = inngest.createFunction(
  { id: "after-interview" },
  { event: "interview/interview.report" },
  async ({ event, step }) => {
    const transcripts = await step.run("transcribe_audio", () => transcribeRecording(event.data.audioUrl));
    const insights = await step.run("generate_insights", () => generateInsights(transcripts.map(t => t.id)));
    return insights;
  }
);

4. Delivering Reports

With Resend, I automated email notifications, sending detailed interview reports directly to candidates and interviewers:

await resend.emails.send({
  from: "[e-mail]",
  to: [candidateEmail, interviewerEmail],
  subject: "Your Interview Report",
  html: render(InterviewReport({ insights })),
});

Challenges

1. Fault Tolerance: Ensuring that recordings were fully processed before transcription required implementing retries and error handling.
2. Scalability: Managing multiple simultaneous interviews while maintaining performance was solved using Convex.dev and serverless functions.
3. Accuracy: AssemblyAI’s robust transcription and summarization capabilities ensured that the generated insights were highly accurate.

Why AssemblyAI?

AssemblyAI was the backbone of this project. Its ease of use, detailed documentation, and unmatched transcription accuracy made it indispensable. The LEMUR feature enabled me to move beyond raw transcripts, creating meaningful, structured insights effortlessly. Without AssemblyAI, this project would have required building complex ML pipelines—a time-consuming and resource-intensive process.

Results

The platform has the potential to:

• Drastically improve the interview experience for candidates.
• Save time for recruiters by automating feedback generation.
• Provide organizations with a consistent framework for evaluating candidates.

Additional Prompts

This project qualifies for the additional prompt related to fault-tolerant workflows:

• By using Inngest, I implemented robust retry mechanisms to ensure seamless execution of transcription and report generation workflows.

Team Submissions

This was a solo submission, but I would like to thank the AssemblyAI team for their incredible support during development.

The project source code can be found here Github repo and remember to check your email inbox for the interview links.

Conclusion

Building this async interview feedback platform was a rewarding experience. By leveraging the power of AssemblyAI, I was able to deliver a solution that is not only innovative but also practical. I encourage everyone to explore AssemblyAI’s capabilities and discover how they can transform your projects.

Thank you for reading, and I hope you enjoy trying out the platform! If you have any questions or feedback, feel free to reach out.