System Design Primer

Imagine you're at a top-tier tech interview, and the interview asks you to design a scalable social media platform for millions of users. Where do you start?

That's the essence of  System Design —the skill that can make or break your career in tech.

Whether you're prepping for a major interview or building scalable applications, System Design skills are essential. It's your software's blueprint to ensure that everything runs smoothly under real-world demands. And top companies use System Design interviews to test how well you can create systems that are efficient, reliable, and scalable.

Today, I'm going to walk you through the critical System Design concepts and strategies you'll need to succeed, whether you're a beginner or an experienced engineer sharpening your skills.

Drawing on my experience at FAANG companies, I'll share practical tips and insights to help you ace your next interview. Here's a look at what I'll cover:

For starters, we'll discuss the first section in this System Design primer—the fundamentals you'll need to master!

System Design fundamentals

Before you can design complex systems, you need a solid foundation in the core components that power them. Operating systems, computer networks, and distributed systems form the backbone of modern software architecture, and mastering these fundamentals will prepare you for even the toughest System Design challenges.

Each of these areas play a critical role:

• Operating systems manage the resources and processes that keep applications running smoothly.
• Computer networks enable communication across machines and locations.
• Distributed systems ensure applications are scalable, reliable, and available.

In the next section, I'll break these concepts down and show you how they come together to create the scalable systems you'll be asked to design in interviews—and beyond.

Of course, there are hundreds of terms and concepts in System Design. However, based on my experience as both a candidate and an interviewer in several System Design interviews, the following topics are considered the highest priority.

Operating systems fundamentals

Operating systems (OS) are more than just the software that runs your applications—they’re the backbone of modern computing. A solid understanding of OS internals, from process management to memory allocation, is essential for any aspiring System Design engineer. Mastering these core concepts can significantly impact the efficiency and reliability of your designs.

Key OS concepts for System Design

• Process management: The ability to handle multiple processes efficiently is at the heart of any OS. Understanding how an OS schedules tasks, manages CPU time, and controls process states will help you create responsive and stable systems.
• File systems: The file system is responsible for storing and retrieving data. Different file systems offer various advantages and trade-offs, affecting performance and scalability. Understanding these can be pivotal when designing systems that handle vast amounts of data.
• OS architectures: Lastly, learn about different OS architectures, such as monolithic and microkernel systems. Each has its own performance implications and trade-offs, which can greatly affect your design choices.

Another fundamental concept closely tied to operating systems is concurrency —the ability to execute multiple tasks simultaneously. concurrency is crucial in modern systems that need to perform several operations at once, whether it's processing user requests, running background tasks, or managing distributed workloads.

In System Design, concurrency is one of the key factors that determines how responsive and scalable your system will be. However, managing concurrent tasks introduces complexity, especially when multiple processes need to access shared resources. Without proper synchronization, you risk issues like race conditions, deadlocks, or inconsistent data.

How concurrency works
Concurrency works by dividing tasks into smaller, independent components that can run in parallel. Synchronization techniques, such as locks, semaphores, and condition variables, act as “traffic lights” that control when processes can safely access shared resources without conflict.

In distributed systems, concurrency becomes even more complex due to network delays and the absence of shared memory. Specialized algorithms like distributed locks and consensus models are required to manage these challenges, ensuring performance and data consistency across multiple machines.

Mastering concurrency not only boosts your system’s efficiency but also prepares you for the design challenges that arise in large-scale, real-world systems.

Continue reading about operating systems with this fundamentals course: Operating Systems: Virtualization, Concurrency & Persistence.

Next up—let's focus on the fundamentals computer networks.

Computer networks fundamentals

Have you ever wondered how computers talk to each other across vast distances? The magic lies in computer networks, interconnected devices sharing information and resources. Think of it as inter-machine communication, much like how processes communicate within a single machine through IPC (inter-process communication).

OSI model and protocols

Computer networks are the backbone of the internet. The OSI (Open Systems Interconnection) model, which divides network tasks into seven layers, ensures everything runs smoothly—from physical wires (Layer 1) to your applications (Layer 7). This layered approach facilitates communication within local area networks (LANs) and across wide area networks (WANs).

TCP (Transmission Control Protocol) and UDP (User Datagram Protocol) show how data travels. TCP ensures reliable, ordered data transfer—ideal for emails and web browsing. UDP offers faster but less reliable communication, perfect for live video streaming.

Application layer protocols

At the top of the OSI model lie application layer protocols like HTTP and HTTPS, the languages your browser uses to talk to websites. HTTP enables web page requests and data exchange, forming the basis of internet communication. HTTPS adds security, ensuring data privacy.

FTP (File Transfer Protocol) and SMTP (Simple Mail Transfer Protocol) facilitate seamless file transfers and email communications, respectively.

APIs and RPCs

APIs revolutionized how systems interact. REST (Representational State Transfer) and GraphQL are popular web API architectures. REST uses a stateless, client-server model, making it scalable and straightforward for public APIs. GraphQL, a more flexible query language, allows clients to request specific data, minimizing over-fetching.

While APIs excel in many areas, remote procedure calls (RPCs) bring another layer of efficiency. RPCs, like Google’s gRPC, allow a program on one computer to execute a function on another as if it were local. It leverages Protocol Buffers and HTTP/2, offering high performance and low latency—perfect for modern microservices.

Communication models

Communication models define how information flows between entities in a system, playing a key role in shaping the design and behavior of technologies we use every day. Two of the most fundamental models are client-server and peer-to-peer (P2P).

In the client-server model, distinct roles exist: clients request services, and servers deliver them. This model, common in web services and databases, simplifies management but introduces a single point of failure if the server goes down.

In contrast, the peer-to-peer (P2P) model decentralizes these roles—each device (peer) acts as both a client and a server. This increases redundancy and resilience, making P2P ideal for use cases like file sharing and blockchain. However, its distributed nature introduces added complexity in management.

Together, these models form the backbone of modern internet architectures, influencing designs like Model-View-Controller (MVC) and  microservices . To dive deeper into how these models function within computer networks, check out the course Grokking Computer Networking for Software Engineers.

Now, onward to distributed systems!

Distributed systems

In today's world, software rarely runs on a single machine. Instead, it operates across multiple computers working together to deliver services. These d istributed systems  share workloads across a network of devices, powering everything from cloud infrastructure to smart home sensors. Their unique characteristics shape how they’re built, maintained, and used.

Key characteristics of distributed systems

These key characteristics define a distributed system’s behavior in the real world:

• Scalability: The main advantage of distributed systems is their capacity to scale. As demands grow, additional computers can be integrated to handle more users, data, or transactions. Scalability can be achieved through vertical scaling (boosting the power of existing servers) or horizontal scaling (adding more servers).
• Availability: Distributed systems are designed for high availability, ensuring they remain functional even if some machines fail. This is achieved through redundancy and fault tolerance, where data and services are replicated across multiple machines.
• Consistency: Maintaining data consistency across numerous machines is challenging. Strict consistency ensures every read sees the latest write but may sacrifice availability. Eventual consistency favors availability, allowing temporary inconsistencies to be resolved. The CAP theorem and PACELC theorem help us understand these trade-offs.
• Latency and performance: Communication between distributed computers introduces latency, affecting performance. Minimizing latency through careful design and optimization is crucial for a responsive user experience.
• Security and privacy: Distributed systems are more susceptible to security threats because they have a larger attack surface (lots of interconnected machines). Robust measures like authentication, authorization, and encryption are essential to protect data and system integrity.
• Monitoring and observability: Monitoring system health and performance is vital for detecting and resolving issues before they impact users. Observability tools provide insights into system behavior, aiding in diagnosis and troubleshooting.
• Resilience and error handling: Failures are inevitable in complex distributed systems. Resilient systems are designed to withstand failures and recover quickly, minimizing downtime and data loss. Effective error handling is key to maintaining reliability.

I can go on with the list, but Dimos Raptis explains the ins and outs of distributed systems really well in his course.

With all that in mind, let's switch gears to talk about System Design interviews, what makes them unique, and how to nail them.

Strategies for acing the System Design interview

Now that we've covered key System Design concepts, let's discuss how to put that knowledge to use in the interview process.

System Design interviews can be challenging because they’re unconventional compared to typical coding interviews. They require not only technical skills but also the ability to structure your thoughts clearly and tackle open-ended problems. With practice and preparation, however, you can develop a strategy to approach these interviews with confidence.

To help you prepare for your System Design interview, I recommend checking out my blog on what NOT to do in a Netflix System Design interview.

Defining system requirements

Before diving into design, the first and most important step is to clarify the system’s requirements. This is crucial for success, as it sets the foundation for everything that follows.

• Functional requirements: These define what the system should do. For example, a social media platform might require features like user profiles, messaging, and posting capabilities.
• Non-functional requirements: These outline how well the system performs. For instance, the same social media platform might need to support 10,000 concurrent users with a response time under 2 seconds, emphasizing performance, reliability, and security.

Remember, functional requirements describe what the system does, while non-functional requirements describe how well it performs those tasks. You’ll need to address both in your interview, demonstrating a clear understanding of core functionalities before considering trade-offs in performance or scalability.

Estimation

Effective System Design includes estimating the system’s scale. You’ll need to calculate factors like queries per second and data storage to ensure the system can handle its expected load. For instance, a platform like X (formerly Twitter) needs to estimate storage, bandwidth, and post views based on user activity.

• Example: Assuming each post with metadata is 30 bytes, 100 million new posts per day would need about 30GB/per day.

Interview tip: The most important estimations an interviewer is looking for are storage, bandwidth, and the number of servers required.

High-level design

Once you’ve defined the requirements, it’s time to start outlining the high-level design. This stage involves identifying the main components and how they interact, without diving into implementation details just yet.
Considerations for high-level design:

• What are the key building blocks of the design?
• SQL databases vs. NoSQL databases?
• What is the database schema/data model?
• What database should we use? Amazon RDS? MongoDB?
• What do the interfaces look like for these components? Do we need REST APIs or RPCs?

For instance, X’s system must efficiently handle high storage and bandwidth requirements. We can see a sample high-level design for X below:

Interview tip: The GitHub repo contains the most-asked System Design interview questions.

Detailed design

Next, dive into the details of each component, ensuring that your design addresses both functional and non-functional requirements.

• Consider different approaches and evaluate their pros and cons.
• Perform a trade-off analysis for caching, load balancing, and data partitioning.
• Evaluate whether to use open-source solutions or custom, closed-source components.

Interview tip:  Check out Twitter/X's detailed system design to see a robust example.

Identifying and addressing bottlenecks

With your detailed design in place, the next crucial step is to identify and mitigate system bottlenecks. Bottlenecks can arise from various aspects, such as traffic, data handling, storage, availability, redundancy, and backups. Addressing these can significantly improve system performance and reliability.
Here are some sample questions to guide you in this process:

• Is there a single point of failure in the system? If so, how can it be eliminated?
• Is data replication sufficient to ensure continued service if a few servers fail?
• Do we have enough copies of our services to avoid potential shutdowns?

By thoroughly analyzing these factors, you can strengthen your system’s resilience and ensure it operates smoothly under various conditions.

Next steps to master System Design

Mastering System Design starts with the fundamentals—operating systems, networking, and distributed systems—and extends into real-world problem-solving. These concepts give you the foundation you need to tackle System Design interviews.

In interviews, clearly define both functional and non-functional requirements. Focus on balancing trade-offs and designing solutions that meet constraints like time and resources. Success in System Design is about more than just code—it's about problem-solving, making smart decisions, and delivering reliable, scalable systems.

To sharpen your skills, explore courses like Grokking Modern System Design Interview, and test yourself with mock interviews to prepare for real-world challenges.

Hopefully this System Design primer has you feeling confident about building your skills and how to show them off in interviews. Good luck—you've got this!