What is gRPC? How does it differ from using a message broker? And what are the perfect use cases for each?
Microservices architecture allows large software systems to be broken down into smaller independent units called services. These services usually come with their own servers and databases. The services in a microservices system need a way to effective way to communicate such that the operations on one service does not affect other services.
There are several ways to connect services together. If you haven’t designed a microservices system before, the first thing that may come to mind is to create REST endpoints for one service that other services can call. While this can work in systems with a few services, it’s not scalable in larger systems. This is because REST works based on a blocking request-response mechanism.
A better way to connect microservices is to use a protocol that offers faster request times or use a non-blocking mechanism to get tasks done. Two technologies that enable this are gRPC and message brokers. gRPC sends binary data over the network and so has a faster request time. But gRPC cannot always be used because it requires memory to be allocated on the receiving end to handle response. One way to mitigate such memory allocation is to use a message broker like Memphis. A message broker allows you to queue messages that will be processed by different services in a microservices system.
This article goes over the similarities and differences between gRPC and message brokers. You will learn about the pros, cons, and ideal use cases of both technologies.
What is gPRC
gRPC (which is short for gRPC Remote Procedural Call) is a communication protocol that is used in place of REST to call functions between a client and a server. The client and the server can be microservices, mobile applications, or CLI tools.
For a gRPC set up to work, the has to be a client and a server. The client will make a proto request to the server and the server responds with a proto response.
image source: https://grpc.io
gRPC uses HTTP 2.0 which is a faster than HTTP 1.1 that REST depends on. HTTP 2.0 enforces a binary format by default. This means protocols using HTTP 2.0 need to serialize their data to binary before sending requests over the network. gRPC uses Protobuf, a binary serialization format, for defining data schema.
gRPC also supports data streaming which allows a single request to return a lot more data than it would typically be in the case of REST. This data streaming can either be a server to client streaming or bi-direction streaming between client to server. Note that the service called the client is the service that makes the initial request while the server is the service that sends the response.
What is a Message Broker
A message broker is a server that contains a persistent, append-only log that stores messages. Think of the message broker as a server that contains different log files that get updated as new data comes in.
An example use case of is an image processing pipeline that converts a large image into smaller images of various sizes. The conversion task takes an average of 10 seconds per image, but you have a thousand users trying to convert their images into different sizes. You can store each conversion task in a queue within the message broker and the message broker sends the tasks to the conversion server. This process prevents the server from being overwhelmed and keeps your services fast.
There are several message brokers on the market and your choice of a message broker will depend on your use case. If you’d prefer a cloud-native message broker, then Memphis is a perfect choice. You can also consider brokers such as Apache Kafka, Redis, and RabbitMQ.
Similarities between gRPC and Message Brokers
Message Format
gRPC has similar features to message brokers, the most prominent being the message format. Both gRPC and Memphis for example use proto3 data serialization format. The data is serialized to binary and sent over the network to the client. When the data reaches the consuming client, it is deserialized back to a form the client can use, like JSON.
Streaming support
gRPC and message brokers also support streaming. This means data can be sent from server to client as soon as the data is produced. An example of this is sending resized images from the image resizer service to the image watermarking service as soon as the image is resized. The image data can either be queued in a message broker or sent in an RPC call. In either case, the image data is streamed from the image resizer service to the image watermarking service.
Language agnostic
gRPC and message brokers are mostly language agnostic, with support in a variety of languages. You can use gRPC in the most widely used languages like Python, JavaScript, Java, Go, and C#. You can also connect to a message broker like Memphis using SDKs in Python, JavaScript, and Go.
How gPRC differs from a message broker
While gRPC has similar use cases as message brokers, they differ in so many other ways. A message broker typically stores its data on a disk while gRPC operates on the RAM. A message broker is installed as an executable on a server while gRPC depends on HTTP 2.0. This section goes into detail on how gRPC differs from a message broker.
Disk storage and Partitioning
A message broker serves as a persistent log data structure store, and so, it works on main memory or disk storage. This allows messages to be recovered if there is a server outage. A message broker like Memphis stores data in “stations” and these stations are partitioned across multiple brokers. This distributed storage increases the fault tolerance of the system.
gRPC on the other hand works with RAM because it operates at the source code layer. This also means that gRPC calls are not persisted to disk. Ideally, gRPC can be combined with a message broker to increase the overall performance of a distributed system.
Stream buffering
gRPC being a messaging protocol cannot buffer streaming data. This is unlike message brokers that can store millions of streamed messages as they are produced. An example scenario is streaming temperature data from a thermometer in a factory. If real-time processing is required, there has to be some server processing the data as it comes. This streaming process can overwhelm the server and so, there needs to be a way to process streams in real-time without overwhelming the server. A message broker is the ideal tool to handle this situation.
Deployment method
gRPC is a protocol based on HTTP 2.0 and works at the runtime layer of the software stack. Popular language runtimes like Node.js, Python, and Java 8 have already implemented HTTP 2.0 and so, support gRPC. A software library is usually used to enable gRPC connections within a language runtime.
A message broker on the other hand is an executable that is installed on a server and uses the memory and disk space to function. Memphis for example can be run on Docker containers or on Kubernetes. Clients that connect to a message broker can use REST APIs or gRPC. A user will build an SDK in most cases to ease the process of interacting with a message broker. Memphis for example has language support in Go, Python, and JavaScript.
Conclusion
While both gRPC and message brokers are used for inter-service communication. gRPC is more suited for inter-service calls while message brokers are more suited for task queueing.
The Memphis message broker is the perfect go-to broker for your inter-service communication needs. Memphis is easy to deploy and easy to use. You can immediately start using Memphis if you have Docker installed. Head over to the quick start to start using Memphis today.
Special thanks to Idan Asulin for the article
Top comments (4)
Cool article I wrote one on gRPC too.
What is gRPC? How to implement gRPC in Node.js - SPRKL
Explore the advantages of gRPC, and the difference between gRPC libraries. Learn how to get gRPC working with Node.js.
Cool!!! thanks for sharing! awesome article :)
It seems you forgot to mention a technology that combines the best of both worlds, which is for example Core NATS (not NATS Streaming aka Jetstream), no I mean that tiny "nerves layer", that does not know message persistence (it would be rather silly to queue requests, right?) But it does offer async request/reply, location transparency and authenication termination, if so desired. Now, I do not have a stake in NATS, so I am most curious if there are any similar technologies out there. Thanks for the article!
NATS is a great example. BTW, Memphis is based on NATS