Introduction to Event Driven Architecture

Event-driven architecture has gained significant popularity in recent years. Advancements in technologies like message brokers and distributed systems have enabled architects to design software in an event-driven manner. But what exactly is event-driven architecture, and why is it so impactful? To understand its purpose and benefits, we first need to review the traditional request-response model.

Traditional Request-Response Model

Let’s take a basic example of a ticket reservation web application. Users log in to purchase tickets for an event they want to attend. The browser collects information from the user and sends a request to the server. The application then performs the following operations in sequence:

1. Checks if the requested number of tickets is available.
2. Locks the tickets for the user, so no one else can request the same tickets simultaneously.
3. Sends a request to a payment service and waits for a response confirming the payment.
4. Changes the status of the tickets to "Acquired."
5. Assigns the tickets to the user’s account.
6. Finally, returns a response to the user with the ticket details and confirmation.

In a typical distributed microservices environment, this flow might look like this:

We assume that two microservices are running on our backend. The Tickets microservice accepts the user’s HTTP request, queries a Mongo database to check ticket availability, and updates the ticket status to "Pending". It then sends a payment order to the Payment microservice over HTTP. The Payment service issues a payment request to a third-party payment gateway provider. Once confirmed, it notifies the Tickets service, which finalizes the transaction by updating the database and assigning the tickets to the user. Finally, the Tickets service responds to the user with a confirmation.

All these operations must be completed for the ticket to be successfully reserved.

While this approach works well in theory and often in practice, a few challenges arise:

1. User wait time: The user must wait for all operations to complete before receiving a response.
2. Thread blocking: In a multithreaded environment, each thread is locked for several seconds or minutes while waiting for the processes to complete. This can cause issues during periods of high traffic, such as when tickets are first released, potentially leading to thread exhaustion.
3. Unpredictable latency: The payment gateway, especially if it involves user authentication via OTP, can introduce unpredictable delays.

For small-scale applications, these issues may not be critical, and adopting an alternative approach could seem unnecessary. However, as our application scales, latency and performance bottlenecks can emerge.

We could reduce User wait time by responding to the user before sending the payment request.

While this approach solves one problem, the server still remains blocked until the entire process completes, meaning it doesn’t solve the thread-blocking issue. We are still processing the request synchronously.

Another approach to mitigate this is horizontal scaling. Horizontally scaling microservices might alleviate some issues, but it introduces cost concerns. Additionally, it doesn’t fully address database congestion, especially when dealing with large datasets or additional operations like checking a user's age for an age-restricted event or updating loyalty points. Also, we can only scale our own microservices but we have no control over third party service providers.

The flow becomes more complex if we want to add new operations. Suppose we wanted to add a recommendation service that suggests new events based on the user's previous selections. Now, we need to send another request to a Recommendation microservice. Each new operation adds complexity and tight coupling, leading to a highly interconnected application.

This is where event-driven architecture becomes a game-changer.

What is an Event

An event represents a fact, action or a state change. Imagine you're a manager at a financial company. One morning, you ask an employee, "I need a report on the quarterly income for Company X". The employee replies, "It will be ready in two hours". You go to your desk and proceed with your daily tasks, eventually forgetting about the report until you receive an email from the employee two hours later with the requested report. This scenario can be considered event-driven; you didn’t wait idly for the task to complete.

Asking the employee for the report can be imagined as triggering an event.

An event is always immutable. Events cannot be changed once they are sent. Events can also be stored indefinitely, and can be consumed by multiple consumers concurrently.

What is a Message Broker

A message broker is a piece of intermediary software between applications and services, allowing them to exchange messages and communicate seamlessly. Unlike protocols like HTTP, message brokers format the message and store it in a message queue. This queue acts as a buffer, storing messages until consumed by the receiving application. The key components of a An event-driven model are:

Message Producers produce the event (send the message to the message queue).

Message Queues store the message

Message Consumers retrieve and process messages from the message queue. Multiple consumers can read the message concurrently from the queue.

Message Brokers Facilitate communication between producers and consumers, adding features like message routing, filtering, delivery acknowledgment, and transformation.

Applying Event-Driven Architecture to Our Example

Now, let’s revisit our ticket reservation web application. If we could notify the user that we’re processing their request and promise to inform them once processing is complete via email or notification, we could solve the earlier issues and process the logic asynchronously.

To reconstruct our ticket reservation flow as event-driven, let’s break down each step to understand how the new model operates.

Step 1: Ticket Reservation Request
The user sends the request to an API gateway. The Request is processed by the Tickets service. The service queries the database to check the tickets' availability and change the tickets status to "Pending". The service then sends a message to the message queue. The message can be formatted as JSON or any other formatting technology according to the message broker technology. Finally, the server responds to the user saying that his reservation is "Accepted" and being processed.

The above diagram may seem unusual since nothing yet comes out of the message queue. However, this illustrates a core concept in event-driven architecture: when a producer sends an event, delivery is only guaranteed within the context of the message queue; which means that the consumers are decoupled from the producers. The broker acknowledges only that the message has been stored , and the producers don’t need to know which service will consume it or how it will be processed. This decoupling enables flexible, scalable systems, as we’ll see later on.

Step 2: Payment Processing
The Payment service consumes the message from the queue, initiates a payment request to the third-party service, and waits for confirmation. Once received, it produces an event indicating successful payment.

Step 3: Ticket Confirmation
The Tickets service consumes the event (Reads the message from the queue), changes the status of the tickets to "Acquired," and completes the reservation process by associating the tickets with the user’s account. Here, the Tickets service acts as a consumer.

Step 4: User Notification
Although the ticket reservation is now finalized, the user still needs to be informed. The final step in the process could involve producing a notification event, which can be sent to an email service or another system to alert the user.

Request-Response vs Event-Driven

Now that we reviewed each approach, let us compare between them and see what we gained.

1. Synchronous vs Asynchronous

In the request-response model, the processing happens synchronously, which means that the user has to wait for the response, and if the server never replies this usually indicates a problem.

In the event driven model, the producer doesn't need to wait for the response or even know how and by who the event will be consumed. The delivery of the message to the consumer is outside of its responsibility and control. This allows the producer to move on to the next task instead of waiting for a response it doesn't actually need.

2. Inversion of Control

In the request-response model, the sender, first, needs to know about the receiver. If the sender needs to communicate with multiple receivers, it needs to send a request to each receiver each with their own unique parameters. This makes the sender depend on the receivers of the request.

In an event-driven model, the producer is not even aware of the consumers of the event. This completely decouples the sender from the receivers.

3. Loose Coupling

The reason event-driven architecture and microservices go very well together is the fact that event-driven architecture changes the dynamics of dependencies dramatically. As our application logic gets more complicated, we can add more and more consumers that consume the same event with the same structure. The producer service is completely decoupled from consumers.

Event-driven architecture offers numerous benefits, changing the way we view traditional applications. However, improper implementation can lead to significant issues. Before switching to such a model, it’s important to assess its suitability for your application. In the next articles, we’ll dive into popular event-driven architectural patterns and when to apply them. Read about the first important design pattern, "The SAGA Pattern," in the next article.