Understanding light-weight threads with Coroutines and Project Loom

#java #projectloom #kotlin #coroutines

Coroutines were added to Kotlin in version 1.3. Since then, people have talked a lot about light-weight threads. In September, Java 21 was released, and with it, was introduced the JEP 444 - Virtual Threads, as part of Project Loom. Both have the same goal, which is to reduce dramatically the effort of writing, maintaining, and observing high-throughput concurrent applications. There are many articles about those subjects, but sometimes, people get confused. How does it work? What is virtual thread? Coroutines and Project loom are the same? My intention with this post is to show you how light-weight threads work and some relations in both projects.

Virtual Threads

To understand how virtual threads work, we should understand the relation between OS Threads and JVM Threads. For each JVM Thread, we need an OS Thread. For instance, when our application needs to process something, it creates a JVM thread that will use an OS Thread. The problem is that sometimes these threads get in a iddle state. Suppose that our application receives a request to save a piece of information in the database. During the process of save, the thread will wait until the database returns. It does not look like a big problem, but according to our application grows, this become a problem. Threads are expensive and limited, so if the number of requests grows, it's necessary more threads. And what happens when the threads achieve the limit? It's necessary to scale the machine or increase the server instance to get more threads. Now can you understand why this is a problem? When the thread waits, we pay a lot just for it. How can we treat this? The correct answer is virtual threads. With virtual threads, we have many JVM threads to one OS thread. The good news is that the virtual thread doesn't need to wait until the finish of an external resource (DB). What occurs is that the virtual thread will be detached from an OS thread and will be in a suspended state. When the process finishes, another virtual thread can get the response. Amazing, right? Rashmin wrote in your article

Because of their lightweight nature, unlike Platform Threads, Virtual Threads are very cheap to create and can be created in very large numbers. So unlike Platform Threads, Thread Pooling is not necessary to handle multiple tasks, a separate virtual thread could be created for each task.

Now that we know how virtual threads work, let's see how we can implement them using Project Loom and Coroutines.

Coroutines

The standard library of Kotlin provides some resources to work with coroutines, but this support is designed in a minimalistic way. There are a few elements, like Continuation, suspendCoroutine, and the suspend keyword. To improve the development, it is a good idea to use kotlinx.coroutines library, which gives us a large number of elements like launch, async, and Deferred. The easiest way to work with coroutines is using runBlocking. Let's see an example

fun main() = runBlocking {
    delay(100L)
}

Inside of runBlocking, we can call suspend functions. But, be careful, according to the documentation

runBlocking runs a new coroutine and blocks the current thread until its completion. This function should not be used from a coroutine. It is designed to bridge regular blocking code to libraries that are written in suspending style, to be used in main functions and in tests.

The better way to work with coroutines is by creating a scope.

suspend fun main(): Person = coroutineScope {
     val user = async { getUserFromDB() }
     val address = async { getAddressFromAPI() }

     return Person(user.await(), address.await())
}

suspend fun getUserFromDB(): User {
     return repository.get()
}

The example above shows us that we just call a suspend function from another suspend function (or from a runBlocking function). From a suspend function I can call not suspend function, but in this way, we lose the power of coroutines. It's important to say that in our example, we needed to explicitly the corotineScope, but working with modern frameworks, like Spring, we can start from a Controller, for instance, with suspend functions, without declaring the scope.

Project Loom

Project Loom aims to bring "easy-to-use, high-throughput, lightweight concurrency" to the JRE. One feature introduced by Project Loom is virtual threads and since Java 21, thanks the JEP444, we can use this kind of code

public static void main(String... x) {
     try (var executor = Executors.newVirtualThreadPerTaskExecutor()) {
    IntStream.range(0, 10_000).forEach(i -> {
        executor.submit(() -> {
            Thread.sleep(Duration.ofSeconds(1));
            return i;
        });
    });
}

The task in this example is simple code — sleep for one second — and modern hardware can easily support 10,000 virtual threads running such code concurrently. Behind the scenes, the JDK runs the code on a small number of OS threads, perhaps as few as one.

Things would be very different if this program used an ExecutorService that creates a new platform thread for each task, such as Executors.newCachedThreadPool(). The ExecutorService would attempt to create 10,000 platform threads, and thus 10,000 OS threads, and the program might crash, depending on the machine and operating system.

Conclusion

Asynchronous programming is a great option to improve the performance of our application. Generally, people use platform threads to work with this paradigm, but as we saw previously, platform threads are very expensive. Thinking about improving that, every day new projects are emerging, like coroutines, project loom, goroutines, and so on. All of them use the concept of light-weight threads, a resource cheaper than traditional threads. In this post, we saw a way to implement this concept with coroutines and project loom. To finish, I hope that you have enjoyed the read, and any doubts, criticism, or points to consider, let me know.

References
https://openjdk.org/jeps/444
https://kotlinlang.org/docs/coroutines-basics.html#your-first-coroutine
https://rashm1n.medium.com/revolutionized-concurrency-introduction-to-java-21-virtual-threads-d563693994d9
https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/run-blocking.html