Comparing WebFlux and Spring MVC with JMeter and Kotlin

1. Introduction

Many times, we need to back up certain claims we make in software development. More often than not, we need to show it very clearly to stakeholders, businesses, and management, the value of our claims. Many times we see a true value in changing some technology, and we see long-term benefits in doing so. Whether the improvements we see are performance-related, code readability, maintainability, or modularity, what actually matters to business are other parameters. These can be anything like fast delivery, feature enabling, image, user experience, profits and business growth. In this way, if we need to make an argument on why we should make a step towards a new technology, we need to be able to justify that. Many times this is not an easy task. We have to clearly identify the business goals and make it clear that our suggestions will make remarkable improvements.
For some time I’ve seen talks about reactive programming and my focus has always been WebFlux. RxJava is on the horizon but that will be the subject of another article.
In this article, I assume a few things about the readers. Experience and knowledge about MVC and reactive architectures are very important. Therefore the basics are very important. We will go through the implementation of two Spring boot services. One follows a blocking Spring MVC architecture and the other one follows a Spring WebFlux Reactive implementation in MVC. For the purpose of simplifying semantics, I’ll be using MVC to refer to the blocking MVC model and WebFlux to refer to the reactive MVC model. These implementations are all built with Kotlin. The unit tests are implemented with Spock and the language used is Groovy. These languages are very similar to Java and so if you don’t know them, but you do know Java, then you can be very much assured that you can fully understand what’s happening.
To keep matters simple, in this article, although we are going to look at some differences between the implementation of a WebFlux application and an MVC application, our tests will only be focused on the artist table. As a follow up to this article I will further explain the rest of the application and further expand the example. For now let’s focus on version 1.0.0.
Let’s imagine that we have to present a case in favour of Reactive Programming. We want to suggest migrating our old blocking MVC application to a reactive non-blocking MVC WebFlux application. We explain everything we know about non-blocking applications. We explain the magic of not blocking the main thread. We explain how this will in the end improve performance and how wonderful it will be to be able to process more requests. In our project there are different players and very often, the gap between the engineers and business needs adjustment. This is where benchmarking comes in. What we need to do in these cases is to clearly show business the benefits of changing to WebFlux. Talking about the servlet thread, multithreading, the subscriber pattern, the spawning of different threads per subscriber, vertical scaling, and JVM, can be a good approach. However, most of the time, what we need is to actually be able to point out in the shortest time possible the benefits of doing this. How do we do this? That’s what we are going to see in this article. Just before we continue, please consider the difference between thinking about how many requests we can get per minute and how long a request takes. We will see later on, why is this so important to keep in mind just now.
Reactive programming with WebFlux allows us to scale vertically up, from within the VM with small independent threads. Another very important aspect of this kind of application is that its producers don’t overwhelm consumers. In other words, given that it’s all event-driven, the small threads that are created by the publishers will trigger the subscriber when needed. Finally, it’s important to note that with Webflux, our programming style is declarative and not imperative. This means that our code doesn’t get executed in the main thread which normally allows us to debug our code sequentially.

2. Development

2.1. Environment setup

In order to be able to run this application and perform the benchmarking tests. Aside from the basic setup, we need to have JMeter installed. We will go through that in this article.
We will be looking at the implementation of two Spring boot applications. One will be implemented with the old MVC model and the other with a new WebFlux model. This is how it looks after running docker-compose:

2.2. Structure

Let’s now have a look at the root modules and folders of the main project we will be looking at:

• concert-demos-gui — not available for this project yet. This project will be completed in the follow-up article where we will dive deeper into WebFlux and its paradigms.
• concert-demos-rest-service-mvc — Our Spring Boot MVC application.
• concert-demos-rest-service-webflux — Our WebFlux application.
• concerts-demos-data — Data repo where our DTO (Data Transfer Objects) reside. This is where our REST contract is defined and its common for both applications.
• docker-psql — The definition and files for our custom Postgres image. Here we will define two databases: concertsdb-mvc and concertsdb-webflux. They are separate in order to make more reliable test results.
• jmeter — All JMeter configuration files location. Here we can also find the test results which will allow us to draw conclusions at the end of this article.

2.3. ER Model

For this article, I’ve created a concert management tool. The features expected are only at least to be able to save an artist on a database. In this case, we will work with two different databases running on our local PostgreSQL server. One database will serve the MVC (Model View Controller) typical architecture and the other database will serve the WebFlux architecture. The latter is one of the many implementations of a reactive programming architecture. Although both databases serve different services, they will share the same model.
Let’s start by examining the ER models of both applications:

As you can see, this is a very simple model in which we start out with a concert. Further, we have a number of listings per concert. When doing this, we are saying that our concert will have listings and each listing can be part of many concerts. This is in other words a many to many relation. A concert listing is essentially a set of an artist, a lyric of one music which we should know by heart during the concert, and a complete list of music the artist will play. Music can also be part of different listings and in fact it can also be sung and played by different artists. This is the reason why the relation between the listing and the music is also a many to many.

2.4. Spring MVC vs WebFlux

In my previous article Reactive Programming applied to Legacy Services — A WebFlux example, I explain a few workarounds to make a fully reactive application connected to SOAP services. Although this may have some use to organizations with legacy software, one important step that the article doesn’t have is a reactive data source. Our data source in that article was only a soap service that we, very simply said, wrapped around the Spring WebFlux architecture. In this article, we are going to actually see a completely reactive Web Application using R2DBC.
Both architectures can follow an Onion model or a Ports and Adapters architecture. We are going to go through an implementation which uses common rules for both architectures. Also, bear in mind that although the complete applications allow us to save concerts, listings, music and artists, our focus for this article is the Artist table. So we are going to focus on the creation and retrieval of artists for both applications.
Perhaps the best way to begin this is just to have a look at the repository layer. Let’s have a look at the MVC implementation (I’m purposely excluding the hashCode and the equals methods because of how long they are. They are also not relevant for this article).

2.4.1 Data Model

First, let’s look at the artist data model :

@Entity
data class Artist(

        @Id @GeneratedValue(strategy = GenerationType.IDENTITY)
        var id: Long? = null,
        val name: String? = null,
        val gender: Gender? = null,
        val careerStart: Long? = null,
        val birthDate: String? = null,
        val birthCity: String? = null,
        val country: String? = null,
        val keywords: String? = null

) {}

In Spring MVC, we have to identify which parameter is the primary key, how the id’s are going to be generated and of course we have to specify that our POJO (Plain Old Java Object) is properly identified by Spring Data JPA(Hibernate inside). This is done by stereotyping our POJO with @Entity and setting up our id annotated with @id and @GeneratedValue.
Let’s compare this implementation with the WebFlux implementation:
import org.jesperancinha.concerts.types.Gender
import org.springframework.data.annotation.Id

data class Artist(
        @Id var id: Long? = null,
        val name: String,
        val gender: Gender,
        val careerStart: Long,
        val birthDate: String,
        val birthCity: String,
        val country: String,
        val keywords: String

) {}

The first difference we notice is the absence of the @Entity annotation and we also don’t see any id generation specific annotations. This is just the way R2DBC is currently implemented. In its current state, we will see that many of the annotations we love to use with JPA/Hibernate are just not there. Many reasons can be pointed out to this, but apparently the most important one is that many of the MVC model annotations enter in direct conflict with the Reactive programming principles. We need to make our application as reactive as possible and in this way a couple of important basic things like delegating the id generation to the database native system or not having to specify that our POJO is also an @Entity are important aspects of this. However, probably the most striking difference between the two implementations is how we establish relations between data tables. Next we will have a look at the relation between the Listing and Artist tables. I know I mentioned that the focus of this article was the Artist table only. However, given that this article is also about making a fully reactive application, it is important to understand how we can make ER relations work.

2.4.2. Making ER work

Let’s compare two implementations side by side. First, let’s look at the MVC implementation of the table Listing:

@Entity
data class Listing(
        @Id @GeneratedValue(strategy = GenerationType.IDENTITY)
        var id: Long = 0,

        @OneToOne
        val artist: Artist? = null,

        @OneToOne
        val referenceMusic: Music? = null,

        @ManyToMany(cascade = [CascadeType.ALL])
        @JoinTable(
                name = "listing_music",
                joinColumns = [JoinColumn(name = "music_id")],
                inverseJoinColumns = [JoinColumn(name = "listing_id")]
        )
        var musics: MutableSet<Music> = HashSet(),

        @ManyToMany(mappedBy = "listings")
        val concerts: MutableSet<Concert> = HashSet()
) {}

Let’s focus on two important annotations @OneToOne and @ManyToMany. And now the WebFlux version:

data class Listing(
        @Id var id: Long? = null,
        val artistId: Long,
        val referenceMusicId: Long
) {}

This latter implementation looks pretty simple, but we are missing our relations. This is part of the R2JDB architecture concept. If we need to consider relations, then we will have to consider dependencies. Dependent requests increase latency. They also can potentially mean more selects to the database. However, R2JDB does provide interesting ways of making unblocking dependency relationships. It also provides ways of issuing specific database queries, but we will look at these later. For now it’s only important to note that the annotations which define relations like one to many, one to one, many to many and many to one are not available in the reactive R2JDB way. We have to choose between getting this data from JOIN queries, or on the other hand from a complex publisher structurer.

2.4.3. Repository

Repositories are in either case very simple and straightforward to implement. Let’s have a look at the MVC case:

interface ArtistRepository : CrudRepository<Artist, Long>

And now, the R2DBC case:

interface ArtistRepository : ReactiveCrudRepository<Artist, Long>

We only see a slight difference here in that one repo is a CrudRepository and the other one is a ReactiveCrudRepository. Also notice that they are coming from different packages.

2.4.4. Service

As before, first a look into the MVC service:

@Service
class ArtistServiceImpl(private val artistRepository: ArtistRepository) : ArtistService {

    override fun getAllArtists(): List<ArtistDto>? {
        return artistRepository.findAll().map { toArtistDto(it) }
    }

    override fun createArtist(artist: ArtistDto): ArtistDto {
        return toArtistDto(artistRepository.save(toArtist(artist)))
    }
}

And now the WebFlux case:

@Service
class ArtistServiceImpl(private val artistRepository: ArtistRepository) : ArtistService {

    override fun getAllArtists(): Flux<ArtistDto>? {
        return artistRepository.findAll().map { toArtistDto(it) }
    }

    override fun createArtist(artist: ArtistDto): Mono<ArtistDto> {
        return artistRepository.save(toArtist(artist)).map { toArtistDto(it) }
    }
}

For the moment let’s keep in mind that ArtistDto is a class that is being shared between both app implementations. In the WebFlux implementation we see that instead of returning objects or a list of objects, we are returning Flux and Mono. These are known as publishers. In WebFlux, publishers are also called the event handlers. Before diving into this, let’s look at how controllers are implemented.

2.4.5. Controllers

Finally let’s look at the controllers. First, let’s look at the controller implementation for the MVC case:

@RestController
@RequestMapping("/concerts/data/artists")
class ArtistControllerImpl(private val artistService: ArtistService) : ArtistController {

    override fun getAllArtists(): List<ArtistDto>? {
        return artistService.getAllArtists()
    }

    override fun createArtist(@RequestBody artistDto: ArtistDto): ArtistDto {
        return artistService.createArtist(artistDto)
    }
}

And the implementation for the WebFlux case:

@RestController
@RequestMapping("/concerts/data/artists")
class ArtistControllerImpl(private val artistService: ArtistService) : ArtistController {

    override fun getAllArtists(): Flux<ArtistDto>? {
        return artistService.getAllArtists()
    }

    override fun createArtist(@RequestBody artistDto: ArtistDto): Mono<ArtistDto> {
        return artistService.createArtist(artistDto)
    }
}

Given that there is minimal business logic in our service layer, we don’t find anything to be different in our controller implementation. Considering this, we still need to understand how in WebFlux the data is finally sent back to the user. After making a call, the callback method of the controller, sends a publisher back to netty, instead of the plain DTO (Data Transfer Object). We then have netty which subscribes to these publishers and returns the callback data to the user. Our subscribers are Flux and Mono. They get executed after the callback has been made. The point here is to make this process independent of the main thread which normally handles requests in an MVC architecture. It is this difference we are going to explore in this article.

2.4.6 Dtos

Both applications use the same REST contract. The project. For our artist POST and GET endpoints, our REST contract is very simple and it is compatible with messages like the one in the following example:

{
  "name": "Nicky Minaj",
  "gender": "FEMALE",
  "careerStart": 1000,
  "birthDate": "a date",
  "birthCity": "Port of Spain",
  "country": "Trinidad en Tobago",
  "keywords": "Rap"
}

3. Testing and benchmarking

3.1. JMeter in a nutshell

JMeter is very easy to use. It’s available on a standalone version and its main purpose is to perform benchmarking tests. Many companies use it and the most interested parties are usually banks and companies which deal with sensitive information. Banks usually deal with an endless number of transactions worldwide. Fintech technologies very easily have to deal with large amounts of data in their AI (Artificial Intelligence) and ML (Machine Learning) systems. Companies that deal with data from sensors need also to be especially sharp when it comes to performance. We JMeter we can very easily make tests out of the box.
In our case, we are going to create multiple artists and get them at the same time. In other words, our tests will consist of POST requests and GET requests.
Let’s have a look at how we make this happen. Since JMeter is a very easy to use tool, I will just highlight the main points of my JMeter configuration.
First, let’s configure the BlazeMeter Concurrency Thread Group (located in Threads (users)/Concurrency Thread Group):

What we are doing here is a process configuration. There will be up to 1000 threads generated for 1 whole minute and they will concurrently use each sampler and make requests to our services. The concurrency will linearly increase up to 1000.
Let’s now configure our samplers. You can find them in Sampler/Http Request. They are basically all configured in the same way:

Once the GET and POST for both MVC and WebFlux are configured (4 samplers), we still have to configure our post requests. Remember that information is being exchanged via JSON formatted messages and JSON is how our REST service contract is defined. We still need to define that in our post requests.
For each of our POST samplers lets add Http parameters (find these in Config Element/Http Header Manager):

In the Body Data tab we’ll fill out the following value:

{
  "name": "Nicky Minaj",
  "gender": "FEMALE",
  "careerStart": 1000,
  "birthDate": "a date",
  "birthCity": "Port of Spain",
  "country": "Trinidad en Tobago",
  "keywords": "Rap"
}

Having all our POST and GET requests configured, let’s now add graphical and table representations of our results. Let’s have a look at just two of them for now, because they are the ones we need to understand what we can achieve with WebFlux.
Let’s configure the Response Time Graph (find this one in Listeners/Response Time Graph):

With this configuration we will be able to see how long the average request takes.
Next, we’ll configure the Summary Report (Listeners/Summary Report):

This listener will give us a lot of summarized information about our requests, but it will also give us the request count. This will be an indication of how much load our services could process.

3.2. Running our tests

We are now ready to run our unit tests. We will first run the MVC and the WebFlux tests separately and then we will run them together and see if we can reach any conclusion. For each test let’s make the world a better place and create thousands of Nicky Minajs and see how many we can make and how performant the system actually is.
The docker PostgreSQL image is already prepared to run these tests. If we want to run these tests without docker and against another PostgreSQL database we should increase its capacity for 2 important properties:

max_connections = 1500
shared_buffers = 512MB

After this we need to restart our PostgresSQL service.

3.2.1. MVC Results

Let’s first examine the results of the MVC tests:

At this point we can notice that our post requests were blocked at some point. We also see that some requests were blocked up to 20 ms. It is still a barely acceptable response time, but still acceptable.
Let’s see what the summary report says:

Label	Samples	Average	Min	Max	Std. Dev.	Error Per	Throughput	Received KB/sec	Sent KB/sec	Avg. Bytes
MVC Get Artists	1149	10235	78	33906	6867.42	0.261	18.86142	12214.32	2.52	663124.1
MVC Post Artists	750	9812	9	33294	7136.17	0.267	12.38359	4.42	4.99	365.4
TOTAL	1899	10068	9	33906	6977.87	0.263	31.17305	12218.71	7.48	401371.1

At this point, we should just take into account that there have been some negligible amount of errors and that we were able to send 1149 GET requests and 750 POST requests in total.

3.2.2.. WebFlux Results

In the case of Webflux we get something like the following:

At this point we also see that the load seems to have an effect on increasing latency in response times. However, this is a behavior that is interesting to note and it could be related to PostgreSQL itself or other elements in the pipeline between the request and the response. What is important here to notice in the response times, is that there doesn’t seem to be any difference between the first tests we ran for MVC and the WebFlux tests. One striking difference though is that looking at the graph, at no point were the requests blocked. This is already an achievement in favor of WebFlux. Let’s look at the summary report now:

Label	Samples	Average	Min	Max	Std. Dev.	Error Per	Throughput	Received KB/sec	Sent KB/sec	Avg. Bytes
WebFlux GET artists	1444	8952	62	20804	6164.03	6.440	23.85830	7209.95	2.99	309451.4
WebFlux Post artists	1013	8254	6	20088	6155.44	4.837	16.75876	7.39	6.43	451.6
TOTAL	2457	8664	6	20804	6170.09	5.779	40.59547	7217.33	9.41	182053.4

We were able to send 1444 GET requests and 1013 POST requests in total. Let’s make our results more truthful and remove the error resulting requests. In this sense we have 1444–1444 * 0.0644 = 1351 GET Requests and 1013–1013 * 0.0483 = 964 POST requests. If we compare these results with WebFlux we see that for POST requests and for the same case we gained 1351–1149 = 202 GET requests. And finally we got 964–750 = 214/ This is a great gain. Considering that I ran these tests on a very average laptop (Google Chrome Notebook ASUS C302C). This is already an indication of great things to come in a production environment

3.2.3. Mixed Results

Now let’s have a look at what could happen when we run both applications in an environment where instead of using separate databases, they are using the same server. In this case I ran these tests with up to 800 concurrent threads to avoid errors.
This is our result:

If we look at our results, we may conclude that the response times of WebFlux are faster. Unfortunately, if we run these tests multiple times we’ll see that performance changes a lot depending on when these tests are run.
Since we cannot take a lot of conclusions with this results, let’s have a look at the summary table:

Label	Samples	Average	Min	Max	Std. Dev.	Error Per	Throughput	Received KB/sec	Sent KB/sec	Avg. Bytes
WebFlux GET artists	1818	1714	7	22403	2846.73	0.000	39.69346	4512.75	5.31	116418.6
MVC Get Artists	1510	3320	7	16520	3516.68	0.066	33.09589	5812.04	4.42	179826.8
WebFlux Post artists	1394	1425	4	14886	2597.78	0.000	30.97984	10.26	12.49	339.1
MVC Post Artists	1179	2813	5	20344	3265.32	0.000	26.18371	8.69	10.56	340.0
TOTAL	5901	2276	4	22403	3160.81	0.017	128.35795	10282.47	32.26	82030.3

What we get from this table is the same as in the individual cases. In this case the error percentage is negligible. For the GET requests we get an increase of 1818–1510 = 308. And finally for the POST requests we get an increase of 1394–1179 = 215.

4. Conclusion

As we have seen through our experiment, determining whether WebFlux is the right option for a potential architectural change, can be difficult to determine. Fortunately we can easily determine the benefits of such a change by measuring performance.
In this article we have seen a way to easily build a Reactive application with just one table. We have compared response times and capacity between an MVC architecture and a WebFlux architecture.
After measuring the results we couldn’t really defend WebFlux in respect to response times. However, load wise, our WebFlux application was able to process many more requests in one minute.
With these results, we no longer need to present our solution in a very technical way. The only thing we need to do is to explain that a reactive application under the same conditions will be able to process many more requests than an MVC application.

Let’s go through two examples of running MVC applications:

---

Example 1: A banking application is being used by a class of customers that possesses a high volume bank account. It’s an application not very often used, but it means a lot to the bank. It must perform well and it must be available all the time. The average use is about 500 times a week. Would this be an example to use Reactive Programming?

R: Reactive Programming is always a great thing to use. In this case, however, the application is already working. There will probably be no performance improvement. There are just not enough requests to challenge the blocking MVC architecture it’s already running in. There are no benefits and just a lot of work to do. It will be a delight to developers, but an expensive project to business. There is no reason to make this change here.

Example 2: A company which receives massive amounts of data from sensors placed around buildings, needs to be as fast as possible. There is a high demand for performance. The application it is running needs to provide time-series data in a way that their equipment reacts as fast as possible to data changes. Can this example be an example to migrate to a Reactive programming model?

R: Certainly! If we need to choose between staying MVC or going MVC reactive then we should go reactive in this case. As proven by our results, reactive applications can take up much more load and stay responsive for much longer. This is a valid concern for such applications/processes which need to handle so much load simultaneously.

In other words, we have seen the three most important pillars of Reactive Applications in action. We have seen that a reactive implementation such as WebFlux can be much more resilient than a blocking MVC architecture implementation. We have seen that in cases of high load, our application was still responsive. We have also seen how the asynchronous nature of WebFlux works in real-time.
I have placed all the source code of this application in GitHub
I hope that you have enjoyed this article as much as I enjoyed writing it.
I’d love to hear your thoughts on it, so please leave your comments below.
Thanks in advance for your help, and thank you for reading!

5. Resources

• Reactive Manifesto
• Spring Boot Kotlin
• Reactive relational databases with R2DBC and Spring
• R2DBC
• Lesson 11 - Date and Time in Kotlin - Creating and formatting
• Spock Example
• JMeter
• JMeter's Concurrency Thread Group
• JMeter's Plugin Manager
• Blaze Meter
• How to increase the max connections in postgres?
• Where are my postgres *.conf files?