In the ever-evolving world of web development, building scalable, efficient, and maintainable applications is critical. While Next.js is a powerful framework for creating seamless full-stack applications, relying solely on its built-in server for complex workflows can sometimes pose challenges. In this blog, I will share insights from my own experience working on a social media platform for pet enthusiasts, where I realized the importance of separating Next.js frontend and backend operations.
Challenges with Using Next.js as the Server
1. Long-Running Tasks
Next.js servers, particularly when deployed on platforms like Vercel, come with limitations on execution time. For example:
- Free Plan: Tasks must finish within 10 seconds.
- Paid Plans: The maximum time extends to 5-9 minutes.
This constraint makes Next.js unsuitable for long-running operations such as AI-based workflows like text-to-video generation, which are resource-intensive and require longer execution times. AI applications are rapidly gaining traction, and these limitations can hinder their implementation on Next.js servers.
2. Background Tasks, Job Queues, and Cron Jobs
Handling background tasks or setting up job queues is another pain point. When working on RabbitMQ, I realized the limitations of relying on Next.js for such workflows. These limitations can lead to inefficiencies in processing tasks that require asynchronous execution, such as:
- Sending emails in bulk.
- Processing large datasets.
- Scheduling periodic tasks with cron jobs.
This was a significant learning curve for me, as I initially chose Next.js for these operations, only to face issues later.
3. Database Connection Management
Next.js’s serverless architecture spins up and tears down instances for each request, which can lead to frequent database connections. This causes:
- Connection Overheads: Increased latency and potential connection pool exhaustion.
- Workflow Inefficiencies: Applications with heavy database usage suffer the most.
4. Cold Start Latency
Serverless functions in Next.js often experience cold starts, especially if the function hasn’t been invoked for a while. This can lead to:
- Slower response times for users.
- Suboptimal performance for critical API endpoints.
5. No Persistent File System
Next.js’s serverless nature means there is no persistent file system. This makes it challenging to:
- Store temporary files.
- Process and store uploaded media efficiently.
6. WebSocket Limitations
WebSocket-based applications like real-time chats or live updates face challenges with Next.js due to limited connection times. Persistent connections are difficult to maintain, making Next.js less ideal for WebSocket-heavy applications.
7. Microservices and Scalability
Next.js servers are not inherently designed for complex microservices architectures. While they excel in providing high-level components and features, they lack the customization options required for:
- Isolating faults in individual services.
- Scaling backend services independently of the frontend.
Why a Custom Server Is a Better Option
After much trial and error, I discovered the benefits of creating a custom Node.js server for backend operations. Here’s why:
1. Best Tools for the Job
With a custom server, you can choose tools tailored to your specific needs. Whether it’s Express.js, Koa, or Fastify, you have full control over your backend stack.
2. Independent Scaling
Backend teams can scale the server independently based on traffic or workload. This separation allows for:
- Efficient resource allocation.
- Better performance under high load.
3. Fault Isolation
Custom servers enable you to isolate faults within specific microservices, preventing issues from cascading across the entire application.
The Best Solution: Combining Next.js with a Custom Server
For Petreon, the best approach is to leverage Next.js for the frontend and a custom server for backend operations. This hybrid model allows me to use the strengths of both systems while addressing their limitations.
Architecture Overview
Here’s how the architecture is structured:
- Next.js Client: Handles the user interface and communicates with the backend.
- Next.js Server: Used for authentication and other lightweight operations related to user management.
- Custom Server: Manages heavy backend operations like database interaction, long-running tasks, WebSockets, etc.
Implementation with Kinde
Kinde is an excellent platform for managing authentication in this setup. Here’s how you can integrate it:
Next.js Server (Authentication)
const { jwtVerify } = require("@kinde-oss/kinde-node-express");
const jwtVerifier = jwtVerify("https://yourdomain.kinde.com", {
audience: "",
});
app.use(async (req, res, next) => {
try {
await jwtVerifier(req, res, next);
} catch (err) {
console.error("Token validation failed:", err.message);
res.status(401).send("Unauthorized");
}
});
Custom Server
In the backend (custom server), set up protected routes to handle secure operations.
app.get("/api/protected", jwtVerifier, (req, res) => {
res.json({ message: "This is protected data" });
});
Next.js Client
Use the Kinde client-side library to fetch tokens and communicate with the backend.
import { useKindeBrowserClient } from "@kinde-oss/kinde-auth-nextjs";
import { useEffect } from "react";
export default function DashboardPage() {
const { accessTokenRaw } = useKindeBrowserClient();
useEffect(() => {
if (!accessTokenRaw) return;
const fetchData = async () => {
const response = await fetch("http://localhost:5000/api/protected", {
headers: {
Authorization: `Bearer ${accessTokenRaw}`,
},
});
const data = await response.json();
console.log(data);
};
fetchData();
}, [accessTokenRaw]);
return (
<div>
<h1>Welcome to the Dashboard</h1>
</div>
);
}
After setting everything up, I verified the API functionality with both Postman and a real user login. Everything worked as expected, and the documentation proved invaluable throughout the process.
Conclusion
Separating the backend server from Next.js is a strategic decision that enhances scalability, fault isolation, and overall performance. For applications , where AI workflows, WebSockets, and database-intensive operations are critical, this approach provides the flexibility and power needed to succeed.
By combining Next.js’s frontend capabilities with a custom server, you can build applications that are both efficient and scalable. Start by using platforms like Kinde for authentication and connect your frontend and backend seamlessly for the best results.
Building modern web applications is about making the right architectural choices—and this approach ensures that you’re ready to handle any challenges that come your way!
Top comments (0)