How to Use Caffeine with Kotlin Coroutines - Introduction to caffeine-coroutines

Motivation

JVM has a widely used library for implementing caching called Caffeine.

https://github.com/ben-manes/caffeine

Since this library is written in Java, it cannot be used directly on Coroutines.

Caffeine provides Async support (an interface compatible with CompletableFuture), which makes it possible to use it on Coroutines with some effort. However, unless you have a deep understanding of both Caffeine and Coroutines, the learning curve can be quite steep.

In fact, there are questions on Stack Overflow regarding its usage with Coroutines, indicating that some people struggle with it.
https://stackoverflow.com/questions/71666175/what-is-the-preferred-way-how-to-add-caffeine-cache-to-kotlin-with-coroutines

To make it easy for everyone to use, I created a library called caffeine-coroutines.

https://github.com/be-hase/caffeine-coroutines

※ It has also been mentioned in the Caffeine README.

Basic Usage

First things first, let's add it to the dependencies.

implementation("dev.hsbrysk:caffeine-coroutines:{{version}}")

As for usage, there are almost no differences from the original Caffeine. The only new thing to learn is that by calling buildCoroutine, you can obtain a CoroutineCache instance that works on Coroutines.

suspend fun main() {
    val cache: CoroutineCache<String, String> = Caffeine.newBuilder()
        .maximumSize(10_000)
        .expireAfterWrite(Duration.ofMinutes(5))
        .buildCoroutine() // buildCoroutineを使う

    val value = cache.get("key") {
        delay(1000) // suspend functionが使えるようになる
        "value"
    }
    println(value)
}

Of course, the loading cache style is also supported. By passing a loader to buildCoroutine, you can obtain a CoroutineLoadingCache instance.

suspend fun main() {
    val cache: CoroutineLoadingCache<String, String> = Caffeine.newBuilder()
        .maximumSize(10_000)
        .expireAfterWrite(Duration.ofMinutes(5))
        .buildCoroutine { // buildCoroutineを使う
            delay(1000) // suspend functionが使えるようになる
            "value"
        }

    val value = cache.get("key")
    println(value)
}

Philosophy

Caffeine provides Cache and LoadingCache for blocking use cases, and AsyncCache and AsyncLoadingCache for async (CompletableFuture) use cases.

As the name suggests, this library focuses solely on providing CoroutineCache and CoroutineLoadingCache for Coroutines use cases.

The introduction of custom APIs/interfaces is kept to a minimum.

Introducing unique APIs/interfaces can confuse users, making adoption difficult and causing trouble when discontinuing usage.

Comparison with Aedile (Similar Implementations)

There is actually another Kotlin wrapper for Caffeine called Aedile.

However, when I implemented caffeine-coroutines, I encountered the following issues with Aedile:
(Many of these issues have been resolved in the recently released version 2, so using Aedile is also an option.)

• Issues with Coroutine Scope Handling

  • Ideally, it should execute in a scope derived from the caller's scope (otherwise, for example, the Coroutine Context will not be inherited), but for some reason, it used a separate scope.
  • As a result, there were issues with inheriting things like MDC.

• Issues with Coroutine Cancellation

  • Although the scope handling issue mentioned above was addressed in later versions, there was an issue where an exception occurring inside the loader would cancel the calling Coroutine.
  • This was fixed in version 2, but the fix seems a bit odd… (simply using the coroutineScope function to create a child scope should suffice).

• Too Many Custom APIs

  • To create a cache, you had to use a custom builder called caffeineBuilder.
  • It’s easier to understand if you can use the official builder as is, and using the custom builder meant that some features available in the original implementation were unavailable.
  • This has been resolved in version 2 by adopting a style similar to caffeine-coroutines.
  • Additionally, metrics required using a custom API called CacheMetrics, which I also found problematic. This has also been deprecated in version 2.

Summary

caffeine-coroutines is a library that makes it easy to use Caffeine on Coroutines. By using the buildCoroutine extension function, you can utilize Caffeine’s features seamlessly in a Coroutines environment.

Although Aedile has improved, it previously had issues such as scope handling, cancellation behavior, and excessive custom APIs. caffeine-coroutines focuses on a simple design that leverages official APIs.

If you want to use Caffeine in a Coroutines environment, give it a try!